Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1ixlNh-0001ed-TO for pgsql-bugs@arkaria.postgresql.org; Sat, 01 Feb 2020 05:31:46 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1ixlNg-0006Q2-Lw for pgsql-bugs@arkaria.postgresql.org; Sat, 01 Feb 2020 05:31:44 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1ixlNg-0006Oh-9y; Sat, 01 Feb 2020 05:31:44 +0000 Received: from mail.thelabyrinth.net ([45.56.70.56]) by makus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1ixlNd-0002dT-R9; Sat, 01 Feb 2020 05:31:43 +0000 Received: from Davids-MacBook.local (unknown [179.61.196.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: dsteele) by mail.thelabyrinth.net (Postfix) with ESMTPSA id 9FAF954663; Sat, 1 Feb 2020 05:31:40 +0000 (UTC) Subject: Re: Don't try fetching future segment of a TLI. To: Kyotaro Horiguchi , pgsql-hackers@lists.postgresql.org Cc: psuderevsky@gmail.com, pgsql-bugs@lists.postgresql.org References: <16159-f5a34a3a04dc67e0@postgresql.org> <20191212.124856.202839248708529678.horikyota.ntt@gmail.com> <20200129.120222.1476610231001551715.horikyota.ntt@gmail.com> From: David Steele Message-ID: <8c831a30-a3a2-de89-2059-e7ce842d7aed@pgmasters.net> Date: Fri, 31 Jan 2020 21:31:40 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:68.0) Gecko/20100101 Thunderbird/68.3.1 MIME-Version: 1.0 In-Reply-To: <20200129.120222.1476610231001551715.horikyota.ntt@gmail.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On 1/28/20 8:02 PM, Kyotaro Horiguchi wrote: > At Tue, 28 Jan 2020 19:13:32 +0300, Pavel Suderevsky >> Regading influence: issue is not about the large amount of WALs to apply >> but in searching for the non-existing WALs on the remote storage, each such >> search can take 5-10 seconds while obtaining existing WAL takes >> milliseconds. > > Wow. I didn't know of a file system that takes that much seconds to > trying non-existent files. Although I still think this is not a bug, > but avoiding that actually leads to a big win on such systems. I have not tested this case but I can imagine it would be slow in practice. It's axiomatic that is hard to prove a negative. With multi-region replication it might well take some time to be sure that the file *really* doesn't exist and hasn't just been lost in a single region. > After a thought, I think it's safe and effectively doable to let > XLogFileReadAnyTLI() refrain from trying WAL segments of too-high > TLIs. Some garbage archive files out of the range of a timeline might > be seen, for example, after reusing archive directory without clearing > files. However, fetching such garbages just to fail doesn't > contribute durability or reliablity at all, I think. The patch seems sane, the trick will be testing it. Pavel, do you have an environment where you can ensure this is a performance benefit? Regards, -- -David david@pgmasters.net