From: Alvaro Herrera <alvherre@alvh.no-ip.org>
To: Andres Freund <andres@anarazel.de>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Andrew Dunstan <andrew@dunslane.net>
Cc: Bossart, Nathan <bossartn@amazon.com>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Cc: masao.fujii@oss.nttdata.com <masao.fujii@oss.nttdata.com>
Cc: pgsql-hackers@lists.postgresql.org <pgsql-hackers@lists.postgresql.org>
Cc: mengjuan.cmj@alibaba-inc.com <mengjuan.cmj@alibaba-inc.com>
Cc: Jakub.Wartak@tomtom.com <Jakub.Wartak@tomtom.com>
Cc: Ryo Matsumura <matsumura.ryo@fujitsu.com>
Subject: Re: prevent immature WAL streaming
Date: Wed, 13 Oct 2021 15:52:46 -0300
Message-ID: <202110131852.rrxabqmxxoiw@alvherre.pgsql> (raw)
In-Reply-To: <20211013183129.jutbvropp7nkg2kk@alap3.anarazel.de>
On 2021-Oct-13, Andres Freund wrote:
> I added LSNs to the error message:
> not ok 1 - 000000010000000000000002 (0/2002350) differs from 000000010000000000000002 (0/2099600)
>
> It appears that the problem is that inbetween the determination of
> rows_walsize the insert LSN moved to the next segment separately from the
> insertion, presumably due to autovacuum/analayze or such.
> <retries, with log_autovacuum_min_duration_statement=0, log_min_duration_statement=0>
Oh, of course.
> Hm. I guess we can disable autovac. But that's not a great solution, there
> might be WAL files due to catalog access etc too.
Well, we don't expect anything else to happen -- the cluster is
otherwise idle. I think we should do it regardless of any other
changes, just to keep things steadier.
> Seems like it might be worth doing the "filling" of the segment with a loop in
> a DO block instead, where the end condition is to be within some distance to
> the end of the segment? With plenty headroom?
Eh, good idea ... didn't think of that, but it should keep things more
stable under strange conditions.
> Another thing: filling a segment by inserting lots of very tiny rows is pretty
> expensive. Can't we use something a bit wider? Perhaps even emit_message?
I think I realized partway through writing the test that I could use
emit_message instead of using a batched row insert ... so, yeah, we
can use it here also.
--
Álvaro Herrera Valdivia, Chile — https://www.EnterpriseDB.com/
"La verdad no siempre es bonita, pero el hambre de ella sí"
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: alvherre@alvh.no-ip.org, andres@anarazel.de, tgl@sss.pgh.pa.us, andrew@dunslane.net, bossartn@amazon.com, horikyota.ntt@gmail.com, masao.fujii@oss.nttdata.com, pgsql-hackers@lists.postgresql.org, mengjuan.cmj@alibaba-inc.com, Jakub.Wartak@tomtom.com, matsumura.ryo@fujitsu.com
Subject: Re: prevent immature WAL streaming
In-Reply-To: <202110131852.rrxabqmxxoiw@alvherre.pgsql>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox