From: Sergei Kornilov <sk@zsrv.org>
To: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
To: a.lubennikova@postgrespro.ru <a.lubennikova@postgrespro.ru>
Cc: robertmhaas@gmail.com <robertmhaas@gmail.com>
Cc: michael@paquier.xyz <michael@paquier.xyz>
Cc: andres@anarazel.de <andres@anarazel.de>
Cc: peter.eisentraut@2ndquadrant.com <peter.eisentraut@2ndquadrant.com>
Cc: pgsql-hackers@postgresql.org <pgsql-hackers@postgresql.org>
Subject: Re: Allow some recovery parameters to be changed with reload
Date: Wed, 28 Oct 2020 15:02:33 +0300
Message-ID: <4920201603879284@mail.yandex.ru> (raw)
In-Reply-To: <20201022.110026.524650499365127827.horikyota.ntt@gmail.com>
References: <20200809052149.GB17986@paquier.xyz>
<CA+TgmobB4Syv3rh_v_QoyS=hMpw_Hf4JmjeOFQvDEQA8f2TFGw@mail.gmail.com>
<1741c40b-73e3-133b-77e6-05ebc7bf3c0f@postgrespro.ru>
<20201022.110026.524650499365127827.horikyota.ntt@gmail.com>
Hello
Sorry for late response.
>> > ... but what's the corresponding hazard here, exactly? It doesn't seem
>> > that there's any way in which the decision one process makes affects
>> > the decision the other process makes. There's still a race condition:
>> > it's possible for a walsender
>> Did you mean walreceiver here?
>
> It's logical walsender. restore_command is used within
> logical_read_xlog_page() via XLogReadDetermineTimeline().
Still have no idea what's the corresponding hazard here.
>> > to use the old restore_command after the
>> > startup process had already used the new one, or the other way around.
>> > However, it doesn't seem like that should confuse anything inside the
>> > server, and therefore I'm not sure we need to code around it.
>> I came up with following scenario. Let's say we have xlog files 1,2,3
>> in dir1 and files 4,5 in dir2. If startup process had only handled
>> files 1 and 2, before we switched restore_command from reading dir1 to
>> reading dir2, it will fail to find next file. IIUC, it will assume
>> that recovery is done, start server and walreceiver. The walreceiver
>> will fail as well. I don't know, how realistic is this case, though.
>
> That operation is somewhat bogus, if the server is not in standby
> mode. In standby mode, startup waits for the next segment safely.
I think it's pilot error. It is already possible to change anything in restore_command by wrapping real command into some script:
> restore_command = '/bin/restore_wal.sh "%f" "%p"'
And one can simple replace this file with something else with different logic. Or even by using some command with separate own settings. Real world example ( https://github.com/wal-g/wal-g ):
> restore_command = '. /etc/wal-g/WALG_AWS_ENV; wal-g wal-fetch "%f" "%p"'
And it is possible to change the real WAL source in ENV script without changing the restore_command. We can't track this, so I not see new issues here.
>> Sergey, could you please attach this thread to the upcoming CF, if
>> you're going to continue working on it.
Sure, I created one: https://commitfest.postgresql.org/30/2802/>> - How will it interact with possible future optimizations of archive
>> - restore? For example, WAL prefetch [1].
Shouldn't we ask the author of such a patch and not me? In particular, does this patch rely on the restore_command not being changed? Probably some form of synchronisation would be neccesary in infrastructure for parallel executing restore commands. On / off handling of restore_command will most likely be required. I did not review this patch.
regards, Sergei
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: sk@zsrv.org, horikyota.ntt@gmail.com, a.lubennikova@postgrespro.ru, robertmhaas@gmail.com, michael@paquier.xyz, andres@anarazel.de, peter.eisentraut@2ndquadrant.com
Subject: Re: Allow some recovery parameters to be changed with reload
In-Reply-To: <4920201603879284@mail.yandex.ru>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox