pg.ddx.io pgsql-hackers@postgresql.org mailing list archive help / color / mirror / Atom feed From: Alvaro Herrera <alvherre@alvh.no-ip.org>
To: Andres Freund <andres@anarazel.de>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Andrew Dunstan <andrew@dunslane.net>
Cc: Bossart, Nathan <bossartn@amazon.com>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Cc: masao.fujii@oss.nttdata.com <masao.fujii@oss.nttdata.com>
Cc: pgsql-hackers@lists.postgresql.org <pgsql-hackers@lists.postgresql.org>
Cc: mengjuan.cmj@alibaba-inc.com <mengjuan.cmj@alibaba-inc.com>
Cc: Jakub.Wartak@tomtom.com <Jakub.Wartak@tomtom.com>
Cc: Ryo Matsumura <matsumura.ryo@fujitsu.com>
Subject: Re: prevent immature WAL streaming
Date: Wed, 13 Oct 2021 17:42:37 -0300
Message-ID: <202110132042.pe4herrhq4a7@alvherre.pgsql> (raw )
In-Reply-To: <202110131957.cjm4ftp4dtdm@alvherre.pgsql >
This works nicely with the TAP test:
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"Postgres is bloatware by design: it was built to house
PhD theses." (Joey Hellerstein, SIGMOD annual conference 2002)
Attachments:
[text/x-diff] 0001-do-away-with-small-tuple-writing.patch (2.4K, ../202110132042.pe4herrhq4a7@alvherre.pgsql/2-0001-do-away-with-small-tuple-writing.patch)
download | inline diff:
From e9bcdb3b44f36ffa1a2377f252d73ff3d7210fd9 Mon Sep 17 00:00:00 2001
From: Alvaro Herrera <alvherre@alvh.no-ip.org>
Date: Wed, 13 Oct 2021 17:41:31 -0300
Subject: [PATCH] do away with small tuple writing
---
.../recovery/t/026_overwrite_contrecord.pl | 39 +++++++++++--------
1 file changed, 22 insertions(+), 17 deletions(-)
diff --git a/src/test/recovery/t/026_overwrite_contrecord.pl b/src/test/recovery/t/026_overwrite_contrecord.pl
index 1725a97531..b7d3b6e03d 100644
--- a/src/test/recovery/t/026_overwrite_contrecord.pl
+++ b/src/test/recovery/t/026_overwrite_contrecord.pl
@@ -22,27 +22,32 @@ $node->init(allows_streaming => 1);
$node->append_conf('postgresql.conf', 'wal_keep_size=1GB');
$node->start;
-$node->safe_psql('postgres', 'create table filler (a int)');
-# First, measure how many bytes does the insertion of 1000 rows produce
-my $start_lsn =
- $node->safe_psql('postgres', q{select pg_current_wal_insert_lsn() - '0/0'});
-$node->safe_psql('postgres',
- 'insert into filler select * from generate_series(1, 1000)');
-my $end_lsn =
- $node->safe_psql('postgres', q{select pg_current_wal_insert_lsn() - '0/0'});
-my $rows_walsize = $end_lsn - $start_lsn;
+$node->safe_psql('postgres', 'create table filler (a int, b text)');
# Now consume all remaining room in the current WAL segment, leaving
# space enough only for the start of a largish record.
$node->safe_psql(
- 'postgres', qq{
-WITH setting AS (
- SELECT setting::int AS wal_segsize
- FROM pg_settings WHERE name = 'wal_segment_size'
-)
-INSERT INTO filler
-SELECT g FROM setting,
- generate_series(1, 1000 * (wal_segsize - ((pg_current_wal_insert_lsn() - '0/0') % wal_segsize)) / $rows_walsize) g
+ 'postgres', q{
+DO $$
+DECLARE
+ wal_segsize int := setting::int FROM pg_settings WHERE name = 'wal_segment_size';
+ remain int;
+ iters int := 0;
+BEGIN
+ LOOP
+ INSERT into filler
+ select g, repeat(md5(g::text), (random() * 60 + 1)::int)
+ from generate_series(1, 10) g;
+
+ remain := wal_segsize - (pg_current_wal_insert_lsn() - '0/0') % wal_segsize;
+ IF remain < 2 * setting::int from pg_settings where name = 'block_size' THEN
+ RAISE log 'exiting after % iterations, % bytes to end of WAL segment', iters, remain;
+ EXIT;
+ END IF;
+ iters := iters + 1;
+ END LOOP;
+END
+$$;
});
my $initfile = $node->safe_psql('postgres',
--
2.30.2
view thread (121+ messages) latest in thread
Message-ID: <202110132042.pe4herrhq4a7@alvherre.pgsql>
Permalink: ../202110132042.pe4herrhq4a7@alvherre.pgsql/
Also on: postgresql.org/message-id/202110132042.pe4herrhq4a7@alvherre.pgsql
copy link · copy postgr.es · copy git am
reply Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: alvherre@alvh.no-ip.org, andres@anarazel.de, tgl@sss.pgh.pa.us, andrew@dunslane.net, bossartn@amazon.com, horikyota.ntt@gmail.com, masao.fujii@oss.nttdata.com, pgsql-hackers@lists.postgresql.org, mengjuan.cmj@alibaba-inc.com, Jakub.Wartak@tomtom.com, matsumura.ryo@fujitsu.com
Subject: Re: prevent immature WAL streaming
In-Reply-To: <202110132042.pe4herrhq4a7@alvherre.pgsql>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox