public inbox for [email protected]help / color / mirror / Atom feed
[PATCH 06/10] Default to LZ4.. 3+ messages / 3 participants [nested] [flat]
* [PATCH 06/10] Default to LZ4.. @ 2021-03-12 21:35 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 3+ messages in thread From: Justin Pryzby @ 2021-03-12 21:35 UTC (permalink / raw) this is meant to exercise in the CIs, and not meant to be merged --- configure | 6 ++++-- configure.ac | 4 ++-- src/backend/access/transam/xlog.c | 2 +- src/backend/utils/misc/guc.c | 2 +- 4 files changed, 8 insertions(+), 6 deletions(-) diff --git a/configure b/configure index fed440adcf..8d76be00c1 100755 --- a/configure +++ b/configure @@ -1575,7 +1575,7 @@ Optional Packages: --with-system-tzdata=DIR use system time zone data in DIR --without-zlib do not use Zlib - --with-lz4 build with LZ4 support + --without-lz4 build without LZ4 support --with-gnu-ld assume the C compiler uses GNU ld [default=no] --with-ssl=LIB use LIB for SSL/TLS support (openssl) --with-openssl obsolete spelling of --with-ssl=openssl @@ -8598,7 +8598,9 @@ $as_echo "#define USE_LZ4 1" >>confdefs.h esac else - with_lz4=no + with_lz4=yes + +$as_echo "#define USE_LZ4 1" >>confdefs.h fi diff --git a/configure.ac b/configure.ac index 8c454128bb..bfcdc88be0 100644 --- a/configure.ac +++ b/configure.ac @@ -990,8 +990,8 @@ AC_SUBST(with_zlib) # LZ4 # AC_MSG_CHECKING([whether to build with LZ4 support]) -PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support], - [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])]) +PGAC_ARG_BOOL(with, lz4, yes, [build without LZ4 support], + [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build without LZ4 support. (--without-lz4)])]) AC_MSG_RESULT([$with_lz4]) AC_SUBST(with_lz4) diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 984ce39cc7..3657f74de9 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -99,7 +99,7 @@ bool EnableHotStandby = false; bool fullPageWrites = true; bool wal_log_hints = false; bool wal_compression = false; -int wal_compression_method = WAL_COMPRESSION_ZLIB; +int wal_compression_method = WAL_COMPRESSION_LZ4; char *wal_consistency_checking_string = NULL; bool *wal_consistency_checking = NULL; bool wal_init_zero = true; diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c index c37a8313d3..52f9cd0242 100644 --- a/src/backend/utils/misc/guc.c +++ b/src/backend/utils/misc/guc.c @@ -4728,7 +4728,7 @@ static struct config_enum ConfigureNamesEnum[] = NULL }, &wal_compression_method, - WAL_COMPRESSION_ZLIB, wal_compression_options, + WAL_COMPRESSION_LZ4, wal_compression_options, NULL, NULL, NULL }, -- 2.17.0 --jozmn01XJZjDjM3N Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0007-add-wal_compression_method-zstd.patch" ^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: Lowering the minimum value for maintenance_work_mem @ 2024-05-20 04:58 Masahiko Sawada <[email protected]> 2024-05-20 06:05 ` Re: Lowering the minimum value for maintenance_work_mem John Naylor <[email protected]> 0 siblings, 1 reply; 3+ messages in thread From: Masahiko Sawada @ 2024-05-20 04:58 UTC (permalink / raw) To: Andres Freund <[email protected]>; +Cc: pgsql-hackers; John Naylor <[email protected]> On Fri, May 17, 2024 at 5:55 AM Andres Freund <[email protected]> wrote: > > Hi, > > In the subthread at [1] I needed to trigger multiple rounds of index vacuuming > within one vacuum. > > It turns out that with the new dead tuple implementation, that got actually > somewhat expensive. Particularly if all tuples on all pages get deleted, the > representation is just "too dense". Normally that's obviously very good, but > for testing, not so much: > > With the minimum setting of maintenance_work_mem=1024kB, a simple table with > narrow rows, where all rows are deleted, the first cleanup happens after > 3697812 dead tids. The table for that has to be > ~128MB. > > Needing a ~128MB table to be able to test multiple cleanup passes makes it > much more expensive to test and consequently will lead to worse test coverage. > > I think we should consider lowering the minimum setting of > maintenance_work_mem to the minimum of work_mem. +1 for lowering the minimum value of maintenance_work_mem. I've faced the same situation. Even if a shared tidstore is empty, TidStoreMemoryUsage() returns 256kB because it's the minimum segment size of DSA, i.e. DSA_MIN_SEGMENT_SIZE. So we can lower the minimum maintenance_work_mem down to 256kB, from a vacuum perspective. Regards, -- Masahiko Sawada Amazon Web Services: https://aws.amazon.com ^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: Lowering the minimum value for maintenance_work_mem 2024-05-20 04:58 Re: Lowering the minimum value for maintenance_work_mem Masahiko Sawada <[email protected]> @ 2024-05-20 06:05 ` John Naylor <[email protected]> 0 siblings, 0 replies; 3+ messages in thread From: John Naylor @ 2024-05-20 06:05 UTC (permalink / raw) To: Masahiko Sawada <[email protected]>; +Cc: Andres Freund <[email protected]>; pgsql-hackers; John Naylor <[email protected]> On Mon, May 20, 2024 at 11:59 AM Masahiko Sawada <[email protected]> wrote: > > On Fri, May 17, 2024 at 5:55 AM Andres Freund <[email protected]> wrote: > > I think we should consider lowering the minimum setting of > > maintenance_work_mem to the minimum of work_mem. > > +1 for lowering the minimum value of maintenance_work_mem. I've faced > the same situation. > > Even if a shared tidstore is empty, TidStoreMemoryUsage() returns > 256kB because it's the minimum segment size of DSA, i.e. > DSA_MIN_SEGMENT_SIZE. So we can lower the minimum maintenance_work_mem > down to 256kB, from a vacuum perspective. I've verified 256kB works with both local and shared memory with the below commands, and 200k records are enough to cause a second round of index cleanup. I don't think we can go much smaller than that without changing how we size the blocks in the node slab contexts (or when they're created), which is currently somewhat arbitrary. That'll need some thought, at least when we get a use case with work_mem as the limit. set maintenance_work_mem = '256kB'; drop table if exists test; create unlogged table test (a int) with (autovacuum_enabled=false); insert into test (a) select i from generate_series(1,200_000) i; create index on test (a); --create index on test (a); -- toggle for parallel vacuum delete from test; vacuum (verbose) test; Side note: I'm confused why shared memory works at all in this case, since it failed for 1MB init segments until we allowed callers to specify a smaller init size. The overhead for DSA seems to be significant for small sizes, as evidenced from the amount of usable memory: shared: INFO: finished vacuuming "john.public.test": index scans: 56 local: INFO: finished vacuuming "john.public.test": index scans: 2 ^ permalink raw reply [nested|flat] 3+ messages in thread
end of thread, other threads:[~2024-05-20 06:05 UTC | newest] Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2021-03-12 21:35 [PATCH 06/10] Default to LZ4.. Justin Pryzby <[email protected]> 2024-05-20 04:58 Re: Lowering the minimum value for maintenance_work_mem Masahiko Sawada <[email protected]> 2024-05-20 06:05 ` Re: Lowering the minimum value for maintenance_work_mem John Naylor <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox