pg.ddx.io  pgsql-bugs@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: David Steele <david@pgmasters.net>
To: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
To: pgsql-hackers@lists.postgresql.org
Cc: psuderevsky@gmail.com
Cc: pgsql-bugs@lists.postgresql.org
Subject: Re: Don't try fetching future segment of a TLI.
Date: Fri, 31 Jan 2020 21:31:40 -0800
Message-ID: <8c831a30-a3a2-de89-2059-e7ce842d7aed@pgmasters.net> (raw)
In-Reply-To: <20200129.120222.1476610231001551715.horikyota.ntt@gmail.com>
References: <16159-f5a34a3a04dc67e0@postgresql.org>
	<20191212.124856.202839248708529678.horikyota.ntt@gmail.com>
	<CAEBTBzsi8HrTTSC1xwL0otzrwdefRSLuwdQ7LnAs+jQeN+EJ8A@mail.gmail.com>
	<20200129.120222.1476610231001551715.horikyota.ntt@gmail.com>

On 1/28/20 8:02 PM, Kyotaro Horiguchi wrote:
 > At Tue, 28 Jan 2020 19:13:32 +0300, Pavel Suderevsky
 >> Regading influence: issue is not about the large amount of WALs to apply
 >> but in searching for the non-existing WALs on the remote storage, 
each such
 >> search can take 5-10 seconds while obtaining existing WAL takes
 >> milliseconds.
 >
 > Wow. I didn't know of a file system that takes that much seconds to
 > trying non-existent files. Although I still think this is not a bug,
 > but avoiding that actually leads to a big win on such systems.

I have not tested this case but I can imagine it would be slow in 
practice.  It's axiomatic that is hard to prove a negative.  With 
multi-region replication it might well take some time to be sure that 
the file *really* doesn't exist and hasn't just been lost in a single 
region.

 > After a thought, I think it's safe and effectively doable to let
 > XLogFileReadAnyTLI() refrain from trying WAL segments of too-high
 > TLIs.  Some garbage archive files out of the range of a timeline might
 > be seen, for example, after reusing archive directory without clearing
 > files.  However, fetching such garbages just to fail doesn't
 > contribute durability or reliablity at all, I think.

The patch seems sane, the trick will be testing it.

Pavel, do you have an environment where you can ensure this is a 
performance benefit?

Regards,
-- 
-David
david@pgmasters.net





view thread (13+ messages)  latest in thread

Message-ID: <8c831a30-a3a2-de89-2059-e7ce842d7aed@pgmasters.net>
Permalink:  ../8c831a30-a3a2-de89-2059-e7ce842d7aed@pgmasters.net/
Also on:    postgresql.org/message-id/8c831a30-a3a2-de89-2059-e7ce842d7aed@pgmasters.net

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-bugs@postgresql.org
  Cc: david@pgmasters.net, horikyota.ntt@gmail.com, pgsql-hackers@lists.postgresql.org, psuderevsky@gmail.com, pgsql-bugs@lists.postgresql.org
  Subject: Re: Don't try fetching future segment of a TLI.
  In-Reply-To: <8c831a30-a3a2-de89-2059-e7ce842d7aed@pgmasters.net>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox