From: David Steele <david@pgmasters.net>
To: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
To: pgsql-hackers@lists.postgresql.org
Cc: psuderevsky@gmail.com
Cc: pgsql-bugs@lists.postgresql.org
Subject: Re: Don't try fetching future segment of a TLI.
Date: Fri, 31 Jan 2020 21:31:40 -0800
Message-ID: <8c831a30-a3a2-de89-2059-e7ce842d7aed@pgmasters.net> (raw)
In-Reply-To: <20200129.120222.1476610231001551715.horikyota.ntt@gmail.com>
References: <16159-f5a34a3a04dc67e0@postgresql.org>
<20191212.124856.202839248708529678.horikyota.ntt@gmail.com>
<CAEBTBzsi8HrTTSC1xwL0otzrwdefRSLuwdQ7LnAs+jQeN+EJ8A@mail.gmail.com>
<20200129.120222.1476610231001551715.horikyota.ntt@gmail.com>
On 1/28/20 8:02 PM, Kyotaro Horiguchi wrote:
> At Tue, 28 Jan 2020 19:13:32 +0300, Pavel Suderevsky
>> Regading influence: issue is not about the large amount of WALs to apply
>> but in searching for the non-existing WALs on the remote storage,
each such
>> search can take 5-10 seconds while obtaining existing WAL takes
>> milliseconds.
>
> Wow. I didn't know of a file system that takes that much seconds to
> trying non-existent files. Although I still think this is not a bug,
> but avoiding that actually leads to a big win on such systems.
I have not tested this case but I can imagine it would be slow in
practice. It's axiomatic that is hard to prove a negative. With
multi-region replication it might well take some time to be sure that
the file *really* doesn't exist and hasn't just been lost in a single
region.
> After a thought, I think it's safe and effectively doable to let
> XLogFileReadAnyTLI() refrain from trying WAL segments of too-high
> TLIs. Some garbage archive files out of the range of a timeline might
> be seen, for example, after reusing archive directory without clearing
> files. However, fetching such garbages just to fail doesn't
> contribute durability or reliablity at all, I think.
The patch seems sane, the trick will be testing it.
Pavel, do you have an environment where you can ensure this is a
performance benefit?
Regards,
--
-David
david@pgmasters.net
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-bugs@postgresql.org
Cc: david@pgmasters.net, horikyota.ntt@gmail.com, pgsql-hackers@lists.postgresql.org, psuderevsky@gmail.com, pgsql-bugs@lists.postgresql.org
Subject: Re: Don't try fetching future segment of a TLI.
In-Reply-To: <8c831a30-a3a2-de89-2059-e7ce842d7aed@pgmasters.net>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox