pg.ddx.io pgsql-hackers@postgresql.org mailing list archive
help / color / mirror / Atom feedremove pg_standby?
25+ messages / 14 participants
[nested] [flat]
Commits from this thread:
2020-11-25 2fbd786c34 7ef52b5d5d b608645c17 b9a027c53a d965f698dc 272ace098d 796f57d216 doc: Fix typos
* remove pg_standby?
@ 2014-11-04 21:36 Peter Eisentraut <peter_e@gmx.net>
2014-11-08 05:01 ` Re: remove pg_standby? Michael Paquier <michael.paquier@gmail.com>
2014-11-10 15:19 ` Re: remove pg_standby? Magnus Hagander <magnus@hagander.net>
2014-11-10 17:16 ` Re: remove pg_standby? Fujii Masao <masao.fujii@gmail.com>
2014-11-10 17:50 ` Re: remove pg_standby? Joshua D. Drake <jd@commandprompt.com>
0 siblings, 4 replies; 25+ messages in thread
From: Peter Eisentraut @ 2014-11-04 21:36 UTC (permalink / raw)
To: pgsql-hackers
While we're talking about removing old things, is there any use left for
pg_standby?
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: remove pg_standby?
2014-11-04 21:36 remove pg_standby? Peter Eisentraut <peter_e@gmx.net>
@ 2014-11-08 05:01 ` Michael Paquier <michael.paquier@gmail.com>
3 siblings, 0 replies; 25+ messages in thread
From: Michael Paquier @ 2014-11-08 05:01 UTC (permalink / raw)
To: Peter Eisentraut <peter_e@gmx.net>; +Cc: pgsql-hackers
On Wed, Nov 5, 2014 at 6:36 AM, Peter Eisentraut <peter_e@gmx.net> wrote:
> While we're talking about removing old things, is there any use left for
> pg_standby?
+1 for this autumn cleanup.
--
Michael
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: remove pg_standby?
2014-11-04 21:36 remove pg_standby? Peter Eisentraut <peter_e@gmx.net>
@ 2014-11-10 15:19 ` Magnus Hagander <magnus@hagander.net>
3 siblings, 0 replies; 25+ messages in thread
From: Magnus Hagander @ 2014-11-10 15:19 UTC (permalink / raw)
To: Peter Eisentraut <peter_e@gmx.net>; +Cc: pgsql-hackers
On Tue, Nov 4, 2014 at 10:36 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
> While we're talking about removing old things, is there any use left for
> pg_standby?
Was the original reason to keep it around anything other than
backwards compatibility? If not, then it was backwards compatibility
with a version that's not even supported anymore - so strong +1 for
getting rid of it.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: remove pg_standby?
2014-11-04 21:36 remove pg_standby? Peter Eisentraut <peter_e@gmx.net>
@ 2014-11-10 17:16 ` Fujii Masao <masao.fujii@gmail.com>
2014-11-10 17:19 ` Re: remove pg_standby? Magnus Hagander <magnus@hagander.net>
3 siblings, 1 reply; 25+ messages in thread
From: Fujii Masao @ 2014-11-10 17:16 UTC (permalink / raw)
To: Peter Eisentraut <peter_e@gmx.net>; +Cc: pgsql-hackers
On Wed, Nov 5, 2014 at 6:36 AM, Peter Eisentraut <peter_e@gmx.net> wrote:
> While we're talking about removing old things, is there any use left for
> pg_standby?
-1 for removing it. There is still the case where I'd like to use log-shipping
rather than replication. For example, it's the case where I need to
compress WAL files before streaming them via very thin network.
We can set up log-shipping using standby_mode and without using
pg_standby, but it keeps emitting the failure of restore_command while
while there is no WAL activity, and which is bothersome. So I still need
pg_standby for log-shipping.
Regards,
--
Fujii Masao
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: remove pg_standby?
2014-11-04 21:36 remove pg_standby? Peter Eisentraut <peter_e@gmx.net>
2014-11-10 17:16 ` Re: remove pg_standby? Fujii Masao <masao.fujii@gmail.com>
@ 2014-11-10 17:19 ` Magnus Hagander <magnus@hagander.net>
2014-11-10 17:43 ` Re: remove pg_standby? Fujii Masao <masao.fujii@gmail.com>
0 siblings, 1 reply; 25+ messages in thread
From: Magnus Hagander @ 2014-11-10 17:19 UTC (permalink / raw)
To: Fujii Masao <masao.fujii@gmail.com>; +Cc: Peter Eisentraut <peter_e@gmx.net>; pgsql-hackers
On Nov 10, 2014 6:16 PM, "Fujii Masao" <masao.fujii@gmail.com> wrote:
>
> On Wed, Nov 5, 2014 at 6:36 AM, Peter Eisentraut <peter_e@gmx.net> wrote:
> > While we're talking about removing old things, is there any use left for
> > pg_standby?
>
> -1 for removing it. There is still the case where I'd like to use
log-shipping
> rather than replication. For example, it's the case where I need to
> compress WAL files before streaming them via very thin network.
> We can set up log-shipping using standby_mode and without using
> pg_standby, but it keeps emitting the failure of restore_command while
> while there is no WAL activity, and which is bothersome. So I still need
> pg_standby for log-shipping.
I didn't realize that part, but maybe we should fix that instead of keeping
pg_standby around?
(BTW, you can use streaming with compression as well using ssl of course,
but it won't get quite the same levels due to smaller block sizes. And
there are certainly still reasons for file based standbys so we should
definitely not remove that)
/Magnus
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: remove pg_standby?
2014-11-04 21:36 remove pg_standby? Peter Eisentraut <peter_e@gmx.net>
2014-11-10 17:16 ` Re: remove pg_standby? Fujii Masao <masao.fujii@gmail.com>
2014-11-10 17:19 ` Re: remove pg_standby? Magnus Hagander <magnus@hagander.net>
@ 2014-11-10 17:43 ` Fujii Masao <masao.fujii@gmail.com>
0 siblings, 0 replies; 25+ messages in thread
From: Fujii Masao @ 2014-11-10 17:43 UTC (permalink / raw)
To: Magnus Hagander <magnus@hagander.net>; +Cc: Peter Eisentraut <peter_e@gmx.net>; pgsql-hackers
On Tue, Nov 11, 2014 at 2:19 AM, Magnus Hagander <magnus@hagander.net> wrote:
>
> On Nov 10, 2014 6:16 PM, "Fujii Masao" <masao.fujii@gmail.com> wrote:
>>
>> On Wed, Nov 5, 2014 at 6:36 AM, Peter Eisentraut <peter_e@gmx.net> wrote:
>> > While we're talking about removing old things, is there any use left for
>> > pg_standby?
>>
>> -1 for removing it. There is still the case where I'd like to use
>> log-shipping
>> rather than replication. For example, it's the case where I need to
>> compress WAL files before streaming them via very thin network.
>> We can set up log-shipping using standby_mode and without using
>> pg_standby, but it keeps emitting the failure of restore_command while
>> while there is no WAL activity, and which is bothersome. So I still need
>> pg_standby for log-shipping.
>
> I didn't realize that part,
The log-shipping standby using standby_mode tries to execute the
restore_command to restore new WAL file but it fails and the message
meaning the failure is logged if there is no new WAL file. Then
the standby tries to execure the same restore_command after five
seconds. But it fails and the message is logged again. These happen
continuously while no new WAL file is appearing in the standby's
archival area.
> but maybe we should fix that instead of keeping
> pg_standby around?
Yes. Even if we do that, we should announce pg_standby will be
removed and wait for several releases before removing it?
> (BTW, you can use streaming with compression as well using ssl of course,
> but it won't get quite the same levels due to smaller block sizes. And there
> are certainly still reasons for file based standbys so we should definitely
> not remove that)
Yes.
Regards,
--
Fujii Masao
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: remove pg_standby?
2014-11-04 21:36 remove pg_standby? Peter Eisentraut <peter_e@gmx.net>
@ 2014-11-10 17:50 ` Joshua D. Drake <jd@commandprompt.com>
2014-11-10 18:48 ` Re: remove pg_standby? Heikki Linnakangas <hlinnakangas@vmware.com>
3 siblings, 1 reply; 25+ messages in thread
From: Joshua D. Drake @ 2014-11-10 17:50 UTC (permalink / raw)
To: Peter Eisentraut <peter_e@gmx.net>; pgsql-hackers
On 11/04/2014 01:36 PM, Peter Eisentraut wrote:
>
> While we're talking about removing old things, is there any use left for
> pg_standby?
-1.
A lot of people, a lot of customers use log shipping for various
creative and business requirement setups.
JD
--
Command Prompt, Inc. - http://www.commandprompt.com/ 503-667-4564
PostgreSQL Support, Training, Professional Services and Development
High Availability, Oracle Conversion, @cmdpromptinc
"If we send our children to Caesar for their education, we should
not be surprised when they come back as Romans."
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: remove pg_standby?
2014-11-04 21:36 remove pg_standby? Peter Eisentraut <peter_e@gmx.net>
2014-11-10 17:50 ` Re: remove pg_standby? Joshua D. Drake <jd@commandprompt.com>
@ 2014-11-10 18:48 ` Heikki Linnakangas <hlinnakangas@vmware.com>
2014-11-10 18:54 ` Re: remove pg_standby? Magnus Hagander <magnus@hagander.net>
0 siblings, 1 reply; 25+ messages in thread
From: Heikki Linnakangas @ 2014-11-10 18:48 UTC (permalink / raw)
To: Joshua D. Drake <jd@commandprompt.com>; Peter Eisentraut <peter_e@gmx.net>; pgsql-hackers
On 11/10/2014 07:50 PM, Joshua D. Drake wrote:
>
> On 11/04/2014 01:36 PM, Peter Eisentraut wrote:
>>
>> While we're talking about removing old things, is there any use left for
>> pg_standby?
>
> -1.
>
> A lot of people, a lot of customers use log shipping for various
> creative and business requirement setups.
Yes, but do they use pg_standby to implement it? If they do, why?
pg_standby is more configurable than the built-in standby_mode=on. You
can set the sleep time, for example, while standby_mode=on uses a
hard-coded delay of 5 s. And pg_standby has a configurable maximum wait
time. And as Fujii pointed out, the built-in system will print an
annoying message to the log every time it attempts to restore a file.
Nevertheless, 99% of users would probably be happy with the built-in thing.
- Heikki
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: remove pg_standby?
2014-11-04 21:36 remove pg_standby? Peter Eisentraut <peter_e@gmx.net>
2014-11-10 17:50 ` Re: remove pg_standby? Joshua D. Drake <jd@commandprompt.com>
2014-11-10 18:48 ` Re: remove pg_standby? Heikki Linnakangas <hlinnakangas@vmware.com>
@ 2014-11-10 18:54 ` Magnus Hagander <magnus@hagander.net>
2015-02-27 20:00 ` Re: remove pg_standby? Josh Berkus <josh@agliodbs.com>
0 siblings, 1 reply; 25+ messages in thread
From: Magnus Hagander @ 2014-11-10 18:54 UTC (permalink / raw)
To: Heikki Linnakangas <hlinnakangas@vmware.com>; +Cc: Joshua D. Drake <jd@commandprompt.com>; Peter Eisentraut <peter_e@gmx.net>; pgsql-hackers
On Mon, Nov 10, 2014 at 7:48 PM, Heikki Linnakangas
<hlinnakangas@vmware.com> wrote:
> On 11/10/2014 07:50 PM, Joshua D. Drake wrote:
>>
>>
>> On 11/04/2014 01:36 PM, Peter Eisentraut wrote:
>>>
>>>
>>> While we're talking about removing old things, is there any use left for
>>> pg_standby?
>>
>>
>> -1.
>>
>> A lot of people, a lot of customers use log shipping for various
>> creative and business requirement setups.
>
>
> Yes, but do they use pg_standby to implement it? If they do, why?
>
> pg_standby is more configurable than the built-in standby_mode=on. You can
> set the sleep time, for example, while standby_mode=on uses a hard-coded
> delay of 5 s. And pg_standby has a configurable maximum wait time. And as
> Fujii pointed out, the built-in system will print an annoying message to the
> log every time it attempts to restore a file. Nevertheless, 99% of users
> would probably be happy with the built-in thing.
As long as pg_standby has features that are actually useful and that
are not in the built-in system, we shouldn't remove it. We should,
however, try to fix those in the main system so we can get rid of it
after that :)
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: remove pg_standby?
2014-11-04 21:36 remove pg_standby? Peter Eisentraut <peter_e@gmx.net>
2014-11-10 17:50 ` Re: remove pg_standby? Joshua D. Drake <jd@commandprompt.com>
2014-11-10 18:48 ` Re: remove pg_standby? Heikki Linnakangas <hlinnakangas@vmware.com>
2014-11-10 18:54 ` Re: remove pg_standby? Magnus Hagander <magnus@hagander.net>
@ 2015-02-27 20:00 ` Josh Berkus <josh@agliodbs.com>
2015-03-02 09:53 ` Re: remove pg_standby? Fujii Masao <masao.fujii@gmail.com>
0 siblings, 1 reply; 25+ messages in thread
From: Josh Berkus @ 2015-02-27 20:00 UTC (permalink / raw)
To: pgsql-hackers
On 11/10/2014 10:54 AM, Magnus Hagander wrote:
> On Mon, Nov 10, 2014 at 7:48 PM, Heikki Linnakangas
> <hlinnakangas@vmware.com> wrote:
>> pg_standby is more configurable than the built-in standby_mode=on. You can
>> set the sleep time, for example, while standby_mode=on uses a hard-coded
>> delay of 5 s. And pg_standby has a configurable maximum wait time. And as
>> Fujii pointed out, the built-in system will print an annoying message to the
>> log every time it attempts to restore a file. Nevertheless, 99% of users
>> would probably be happy with the built-in thing.
>
> As long as pg_standby has features that are actually useful and that
> are not in the built-in system, we shouldn't remove it. We should,
> however, try to fix those in the main system so we can get rid of it
> after that :)
As of current 9.5, we have configurable retries and standby delay in
mainstream. Is there some reason we still need pg_standby?
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: remove pg_standby?
2014-11-04 21:36 remove pg_standby? Peter Eisentraut <peter_e@gmx.net>
2014-11-10 17:50 ` Re: remove pg_standby? Joshua D. Drake <jd@commandprompt.com>
2014-11-10 18:48 ` Re: remove pg_standby? Heikki Linnakangas <hlinnakangas@vmware.com>
2014-11-10 18:54 ` Re: remove pg_standby? Magnus Hagander <magnus@hagander.net>
2015-02-27 20:00 ` Re: remove pg_standby? Josh Berkus <josh@agliodbs.com>
@ 2015-03-02 09:53 ` Fujii Masao <masao.fujii@gmail.com>
2015-03-02 11:37 ` Re: remove pg_standby? Heikki Linnakangas <hlinnaka@iki.fi>
0 siblings, 1 reply; 25+ messages in thread
From: Fujii Masao @ 2015-03-02 09:53 UTC (permalink / raw)
To: Josh Berkus <josh@agliodbs.com>; +Cc: pgsql-hackers
On Sat, Feb 28, 2015 at 5:00 AM, Josh Berkus <josh@agliodbs.com> wrote:
> On 11/10/2014 10:54 AM, Magnus Hagander wrote:
>> On Mon, Nov 10, 2014 at 7:48 PM, Heikki Linnakangas
>> <hlinnakangas@vmware.com> wrote:
>>> pg_standby is more configurable than the built-in standby_mode=on. You can
>>> set the sleep time, for example, while standby_mode=on uses a hard-coded
>>> delay of 5 s. And pg_standby has a configurable maximum wait time. And as
>>> Fujii pointed out, the built-in system will print an annoying message to the
>>> log every time it attempts to restore a file. Nevertheless, 99% of users
>>> would probably be happy with the built-in thing.
>>
>> As long as pg_standby has features that are actually useful and that
>> are not in the built-in system, we shouldn't remove it. We should,
>> however, try to fix those in the main system so we can get rid of it
>> after that :)
>
> As of current 9.5, we have configurable retries and standby delay in
> mainstream. Is there some reason we still need pg_standby?
Yes, it's not easy to perform "fast failover" without pg_standby for now.
Regards,
--
Fujii Masao
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: remove pg_standby?
2014-11-04 21:36 remove pg_standby? Peter Eisentraut <peter_e@gmx.net>
2014-11-10 17:50 ` Re: remove pg_standby? Joshua D. Drake <jd@commandprompt.com>
2014-11-10 18:48 ` Re: remove pg_standby? Heikki Linnakangas <hlinnakangas@vmware.com>
2014-11-10 18:54 ` Re: remove pg_standby? Magnus Hagander <magnus@hagander.net>
2015-02-27 20:00 ` Re: remove pg_standby? Josh Berkus <josh@agliodbs.com>
2015-03-02 09:53 ` Re: remove pg_standby? Fujii Masao <masao.fujii@gmail.com>
@ 2015-03-02 11:37 ` Heikki Linnakangas <hlinnaka@iki.fi>
2015-03-02 12:04 ` Re: remove pg_standby? Fujii Masao <masao.fujii@gmail.com>
0 siblings, 1 reply; 25+ messages in thread
From: Heikki Linnakangas @ 2015-03-02 11:37 UTC (permalink / raw)
To: Fujii Masao <masao.fujii@gmail.com>; Josh Berkus <josh@agliodbs.com>; +Cc: pgsql-hackers
On 03/02/2015 11:53 AM, Fujii Masao wrote:
> On Sat, Feb 28, 2015 at 5:00 AM, Josh Berkus <josh@agliodbs.com> wrote:
>> On 11/10/2014 10:54 AM, Magnus Hagander wrote:
>>> On Mon, Nov 10, 2014 at 7:48 PM, Heikki Linnakangas
>>> <hlinnakangas@vmware.com> wrote:
>>>> pg_standby is more configurable than the built-in standby_mode=on. You can
>>>> set the sleep time, for example, while standby_mode=on uses a hard-coded
>>>> delay of 5 s. And pg_standby has a configurable maximum wait time. And as
>>>> Fujii pointed out, the built-in system will print an annoying message to the
>>>> log every time it attempts to restore a file. Nevertheless, 99% of users
>>>> would probably be happy with the built-in thing.
>>>
>>> As long as pg_standby has features that are actually useful and that
>>> are not in the built-in system, we shouldn't remove it. We should,
>>> however, try to fix those in the main system so we can get rid of it
>>> after that :)
>>
>> As of current 9.5, we have configurable retries and standby delay in
>> mainstream. Is there some reason we still need pg_standby?
>
> Yes, it's not easy to perform "fast failover" without pg_standby for now.
What is "fast failover"?
- Heikki
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: remove pg_standby?
2014-11-04 21:36 remove pg_standby? Peter Eisentraut <peter_e@gmx.net>
2014-11-10 17:50 ` Re: remove pg_standby? Joshua D. Drake <jd@commandprompt.com>
2014-11-10 18:48 ` Re: remove pg_standby? Heikki Linnakangas <hlinnakangas@vmware.com>
2014-11-10 18:54 ` Re: remove pg_standby? Magnus Hagander <magnus@hagander.net>
2015-02-27 20:00 ` Re: remove pg_standby? Josh Berkus <josh@agliodbs.com>
2015-03-02 09:53 ` Re: remove pg_standby? Fujii Masao <masao.fujii@gmail.com>
2015-03-02 11:37 ` Re: remove pg_standby? Heikki Linnakangas <hlinnaka@iki.fi>
@ 2015-03-02 12:04 ` Fujii Masao <masao.fujii@gmail.com>
2015-03-02 18:07 ` Re: remove pg_standby? Josh Berkus <josh@agliodbs.com>
0 siblings, 1 reply; 25+ messages in thread
From: Fujii Masao @ 2015-03-02 12:04 UTC (permalink / raw)
To: hlinnaka@iki.fi; +Cc: Josh Berkus <josh@agliodbs.com>; pgsql-hackers
On Mon, Mar 2, 2015 at 8:37 PM, Heikki Linnakangas <hlinnaka@iki.fi> wrote:
> On 03/02/2015 11:53 AM, Fujii Masao wrote:
>>
>> On Sat, Feb 28, 2015 at 5:00 AM, Josh Berkus <josh@agliodbs.com> wrote:
>>>
>>> On 11/10/2014 10:54 AM, Magnus Hagander wrote:
>>>>
>>>> On Mon, Nov 10, 2014 at 7:48 PM, Heikki Linnakangas
>>>> <hlinnakangas@vmware.com> wrote:
>>>>>
>>>>> pg_standby is more configurable than the built-in standby_mode=on. You
>>>>> can
>>>>> set the sleep time, for example, while standby_mode=on uses a
>>>>> hard-coded
>>>>> delay of 5 s. And pg_standby has a configurable maximum wait time. And
>>>>> as
>>>>> Fujii pointed out, the built-in system will print an annoying message
>>>>> to the
>>>>> log every time it attempts to restore a file. Nevertheless, 99% of
>>>>> users
>>>>> would probably be happy with the built-in thing.
>>>>
>>>>
>>>> As long as pg_standby has features that are actually useful and that
>>>> are not in the built-in system, we shouldn't remove it. We should,
>>>> however, try to fix those in the main system so we can get rid of it
>>>> after that :)
>>>
>>>
>>> As of current 9.5, we have configurable retries and standby delay in
>>> mainstream. Is there some reason we still need pg_standby?
>>
>>
>> Yes, it's not easy to perform "fast failover" without pg_standby for now.
>
>
> What is "fast failover"?
Per document,
------------------
In fast failover, the server is brought up immediately. Any WAL files
in the archive that have not yet been applied will be ignored, and all
transactions in those files are lost. To trigger a fast failover,
create a trigger file and write the word fast into it. pg_standby can
also be configured to execute a fast failover automatically if no new
WAL file appears within a defined interval.
------------------
Regards,
--
Fujii Masao
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: remove pg_standby?
2014-11-04 21:36 remove pg_standby? Peter Eisentraut <peter_e@gmx.net>
2014-11-10 17:50 ` Re: remove pg_standby? Joshua D. Drake <jd@commandprompt.com>
2014-11-10 18:48 ` Re: remove pg_standby? Heikki Linnakangas <hlinnakangas@vmware.com>
2014-11-10 18:54 ` Re: remove pg_standby? Magnus Hagander <magnus@hagander.net>
2015-02-27 20:00 ` Re: remove pg_standby? Josh Berkus <josh@agliodbs.com>
2015-03-02 09:53 ` Re: remove pg_standby? Fujii Masao <masao.fujii@gmail.com>
2015-03-02 11:37 ` Re: remove pg_standby? Heikki Linnakangas <hlinnaka@iki.fi>
2015-03-02 12:04 ` Re: remove pg_standby? Fujii Masao <masao.fujii@gmail.com>
@ 2015-03-02 18:07 ` Josh Berkus <josh@agliodbs.com>
2015-03-02 23:22 ` Re: remove pg_standby? Fujii Masao <masao.fujii@gmail.com>
0 siblings, 1 reply; 25+ messages in thread
From: Josh Berkus @ 2015-03-02 18:07 UTC (permalink / raw)
To: Fujii Masao <masao.fujii@gmail.com>; hlinnaka@iki.fi; +Cc: pgsql-hackers
> Per document,
>
> ------------------
> In fast failover, the server is brought up immediately. Any WAL files
> in the archive that have not yet been applied will be ignored, and all
> transactions in those files are lost. To trigger a fast failover,
> create a trigger file and write the word fast into it. pg_standby can
> also be configured to execute a fast failover automatically if no new
> WAL file appears within a defined interval.
> ------------------
I thought we had that as a 9.4 feature, actually. Well wait ... that's
for streaming.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: remove pg_standby?
2014-11-04 21:36 remove pg_standby? Peter Eisentraut <peter_e@gmx.net>
2014-11-10 17:50 ` Re: remove pg_standby? Joshua D. Drake <jd@commandprompt.com>
2014-11-10 18:48 ` Re: remove pg_standby? Heikki Linnakangas <hlinnakangas@vmware.com>
2014-11-10 18:54 ` Re: remove pg_standby? Magnus Hagander <magnus@hagander.net>
2015-02-27 20:00 ` Re: remove pg_standby? Josh Berkus <josh@agliodbs.com>
2015-03-02 09:53 ` Re: remove pg_standby? Fujii Masao <masao.fujii@gmail.com>
2015-03-02 11:37 ` Re: remove pg_standby? Heikki Linnakangas <hlinnaka@iki.fi>
2015-03-02 12:04 ` Re: remove pg_standby? Fujii Masao <masao.fujii@gmail.com>
2015-03-02 18:07 ` Re: remove pg_standby? Josh Berkus <josh@agliodbs.com>
@ 2015-03-02 23:22 ` Fujii Masao <masao.fujii@gmail.com>
2015-03-03 22:01 ` Re: remove pg_standby? Robert Haas <robertmhaas@gmail.com>
0 siblings, 1 reply; 25+ messages in thread
From: Fujii Masao @ 2015-03-02 23:22 UTC (permalink / raw)
To: Josh Berkus <josh@agliodbs.com>; +Cc: pgsql-hackers
On Tue, Mar 3, 2015 at 3:07 AM, Josh Berkus <josh@agliodbs.com> wrote:
>
>> Per document,
>>
>> ------------------
>> In fast failover, the server is brought up immediately. Any WAL files
>> in the archive that have not yet been applied will be ignored, and all
>> transactions in those files are lost. To trigger a fast failover,
>> create a trigger file and write the word fast into it. pg_standby can
>> also be configured to execute a fast failover automatically if no new
>> WAL file appears within a defined interval.
>> ------------------
>
> I thought we had that as a 9.4 feature, actually. Well wait ... that's
> for streaming.
s/9.4/9.3?
That's different from one we had in 9.3. Fast failover that pg_standby
supports is something like the feature that I was proposing at
http://www.postgresql.org/message-id/CAHGQGwHtvyDqKZaYWYA9zyyLEcAKiF5P0KpcpuNE_tsrGTFtQw@mail.gmail....
that is, the feature which allows us to give up replaying remaining
WAL data for some reasons at the standby promotion. OTOH, fast
failover that was supported in 9.3 enables us to skip an end-of-recovery
checkpoint at the promotion and reduce the failover time.
Regards,
--
Fujii Masao
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: remove pg_standby?
2014-11-04 21:36 remove pg_standby? Peter Eisentraut <peter_e@gmx.net>
2014-11-10 17:50 ` Re: remove pg_standby? Joshua D. Drake <jd@commandprompt.com>
2014-11-10 18:48 ` Re: remove pg_standby? Heikki Linnakangas <hlinnakangas@vmware.com>
2014-11-10 18:54 ` Re: remove pg_standby? Magnus Hagander <magnus@hagander.net>
2015-02-27 20:00 ` Re: remove pg_standby? Josh Berkus <josh@agliodbs.com>
2015-03-02 09:53 ` Re: remove pg_standby? Fujii Masao <masao.fujii@gmail.com>
2015-03-02 11:37 ` Re: remove pg_standby? Heikki Linnakangas <hlinnaka@iki.fi>
2015-03-02 12:04 ` Re: remove pg_standby? Fujii Masao <masao.fujii@gmail.com>
2015-03-02 18:07 ` Re: remove pg_standby? Josh Berkus <josh@agliodbs.com>
2015-03-02 23:22 ` Re: remove pg_standby? Fujii Masao <masao.fujii@gmail.com>
@ 2015-03-03 22:01 ` Robert Haas <robertmhaas@gmail.com>
0 siblings, 0 replies; 25+ messages in thread
From: Robert Haas @ 2015-03-03 22:01 UTC (permalink / raw)
To: Fujii Masao <masao.fujii@gmail.com>; +Cc: Josh Berkus <josh@agliodbs.com>; hlinnaka <hlinnaka@iki.fi>; pgsql-hackers
On Mon, Mar 2, 2015 at 6:22 PM, Fujii Masao <masao.fujii@gmail.com> wrote:
> On Tue, Mar 3, 2015 at 3:07 AM, Josh Berkus <josh@agliodbs.com> wrote:
>>> Per document,
>>>
>>> ------------------
>>> In fast failover, the server is brought up immediately. Any WAL files
>>> in the archive that have not yet been applied will be ignored, and all
>>> transactions in those files are lost. To trigger a fast failover,
>>> create a trigger file and write the word fast into it. pg_standby can
>>> also be configured to execute a fast failover automatically if no new
>>> WAL file appears within a defined interval.
>>> ------------------
>>
>> I thought we had that as a 9.4 feature, actually. Well wait ... that's
>> for streaming.
>
> s/9.4/9.3?
>
> That's different from one we had in 9.3. Fast failover that pg_standby
> supports is something like the feature that I was proposing at
> http://www.postgresql.org/message-id/CAHGQGwHtvyDqKZaYWYA9zyyLEcAKiF5P0KpcpuNE_tsrGTFtQw@mail.gmail....
> that is, the feature which allows us to give up replaying remaining
> WAL data for some reasons at the standby promotion. OTOH, fast
> failover that was supported in 9.3 enables us to skip an end-of-recovery
> checkpoint at the promotion and reduce the failover time.
Calling those things by the same name is very confusing. The
data-losing feature ought to be called "immediate failover" or
something else more dangerous-sounding than "fast".
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: [PATCH] remove pg_standby
@ 2020-11-21 19:41 Justin Pryzby <pryzby@telsasoft.com>
2020-11-25 09:04 ` Re: [PATCH] remove pg_standby Peter Eisentraut <peter.eisentraut@enterprisedb.com>
0 siblings, 1 reply; 25+ messages in thread
From: Justin Pryzby @ 2020-11-21 19:41 UTC (permalink / raw)
To: Peter Eisentraut <peter.eisentraut@enterprisedb.com>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; Tom Lane <tgl@sss.pgh.pa.us>; Stephen Frost <sfrost@snowman.net>; Ian Lawrence Barwick <barwick@gmail.com>; pgsql-hackers
On Fri, Nov 20, 2020 at 05:26:54PM +0100, Peter Eisentraut wrote:
> On 2020-10-29 03:44, Justin Pryzby wrote:
> > diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
> > index 4e833d79ef..be4292ec33 100644
> > --- a/doc/src/sgml/contrib.sgml
> > +++ b/doc/src/sgml/contrib.sgml
> > @@ -199,6 +199,5 @@ pages.
> > part of the core <productname>PostgreSQL</productname> distribution.
> > </para>
> > - &pgstandby;
> > </sect1>
> > </appendix>
>
> With this removal, that section becomes empty. So you probably want to
> clean up or reorganize this a bit.
>
> See https://www.postgresql.org/docs/devel/contrib-prog.html for the context.
Oops. I guess I'd write something like this. If we just remove it, then
there'd no place to add a new server application, and "client applications"
would be the only subsection.
--
Justin
Attachments: download all (2 files, .tar.gz)
v2-0001-Add-missing-word-are.patch (text/x-diff, 1.3K) download
From 2cc4d8adc0d8e2f7bd4c21989a85e6e97b1d2fd6 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 20 Nov 2020 14:31:15 -0600
Subject: [PATCH v2 1/2] Add missing word: 'are'
Should backpatch
---
doc/src/sgml/contrib.sgml | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index 4e833d79ef..ae2759be55 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -180,7 +180,7 @@ pages.
applications in <literal>contrib</literal>. They can be run from anywhere,
independent of where the database server resides. See
also <xref linkend="reference-client"/> for information about client
- applications that part of the core <productname>PostgreSQL</productname>
+ applications that are part of the core <productname>PostgreSQL</productname>
distribution.
</para>
@@ -196,7 +196,7 @@ pages.
applications in <literal>contrib</literal>. They are typically run on the
host where the database server resides. See also <xref
linkend="reference-server"/> for information about server applications that
- part of the core <productname>PostgreSQL</productname> distribution.
+ are part of the core <productname>PostgreSQL</productname> distribution.
</para>
&pgstandby;
--
2.17.0
v2-0002-Retire-pg_standby.patch (text/x-diff, 43.8K) download
From 626fba9074d6bd0e0d6615c0be5932ed63603455 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Mon, 26 Oct 2020 16:37:46 -0500
Subject: [PATCH v2 2/2] Retire pg_standby..
..since pg9.0, use builtin streaming replication protocol with warm/hot standby
---
contrib/Makefile | 1 -
contrib/pg_standby/.gitignore | 1 -
contrib/pg_standby/Makefile | 20 -
contrib/pg_standby/pg_standby.c | 907 ----------------------------
doc/src/sgml/contrib.sgml | 7 +-
doc/src/sgml/filelist.sgml | 1 -
doc/src/sgml/high-availability.sgml | 14 +-
doc/src/sgml/pgstandby.sgml | 394 ------------
src/tools/msvc/Mkvcbuild.pm | 4 +-
9 files changed, 7 insertions(+), 1342 deletions(-)
delete mode 100644 contrib/pg_standby/.gitignore
delete mode 100644 contrib/pg_standby/Makefile
delete mode 100644 contrib/pg_standby/pg_standby.c
delete mode 100644 doc/src/sgml/pgstandby.sgml
diff --git a/contrib/Makefile b/contrib/Makefile
index 7a4866e338..cdc041c7db 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -33,7 +33,6 @@ SUBDIRS = \
pg_buffercache \
pg_freespacemap \
pg_prewarm \
- pg_standby \
pg_stat_statements \
pg_surgery \
pg_trgm \
diff --git a/contrib/pg_standby/.gitignore b/contrib/pg_standby/.gitignore
deleted file mode 100644
index a401b085a8..0000000000
--- a/contrib/pg_standby/.gitignore
+++ /dev/null
@@ -1 +0,0 @@
-/pg_standby
diff --git a/contrib/pg_standby/Makefile b/contrib/pg_standby/Makefile
deleted file mode 100644
index 87732bedf1..0000000000
--- a/contrib/pg_standby/Makefile
+++ /dev/null
@@ -1,20 +0,0 @@
-# contrib/pg_standby/Makefile
-
-PGFILEDESC = "pg_standby - supports creation of a warm standby"
-PGAPPICON = win32
-
-PROGRAM = pg_standby
-OBJS = \
- $(WIN32RES) \
- pg_standby.o
-
-ifdef USE_PGXS
-PG_CONFIG = pg_config
-PGXS := $(shell $(PG_CONFIG) --pgxs)
-include $(PGXS)
-else
-subdir = contrib/pg_standby
-top_builddir = ../..
-include $(top_builddir)/src/Makefile.global
-include $(top_srcdir)/contrib/contrib-global.mk
-endif
diff --git a/contrib/pg_standby/pg_standby.c b/contrib/pg_standby/pg_standby.c
deleted file mode 100644
index c9f33e4254..0000000000
--- a/contrib/pg_standby/pg_standby.c
+++ /dev/null
@@ -1,907 +0,0 @@
-/*
- * contrib/pg_standby/pg_standby.c
- *
- *
- * pg_standby.c
- *
- * Production-ready example of how to create a Warm Standby
- * database server using continuous archiving as a
- * replication mechanism
- *
- * We separate the parameters for archive and nextWALfile
- * so that we can check the archive exists, even if the
- * WAL file doesn't (yet).
- *
- * This program will be executed once in full for each file
- * requested by the warm standby server.
- *
- * It is designed to cater to a variety of needs, as well
- * providing a customizable section.
- *
- * Original author: Simon Riggs simon@2ndquadrant.com
- * Current maintainer: Simon Riggs
- */
-#include "postgres_fe.h"
-
-#include <ctype.h>
-#include <dirent.h>
-#include <sys/stat.h>
-#include <fcntl.h>
-#include <signal.h>
-#include <sys/time.h>
-
-#include "access/xlog_internal.h"
-#include "pg_getopt.h"
-
-const char *progname;
-
-int WalSegSz = -1;
-
-/* Options and defaults */
-int sleeptime = 5; /* amount of time to sleep between file checks */
-int waittime = -1; /* how long we have been waiting, -1 no wait
- * yet */
-int maxwaittime = 0; /* how long are we prepared to wait for? */
-int keepfiles = 0; /* number of WAL files to keep, 0 keep all */
-int maxretries = 3; /* number of retries on restore command */
-bool debug = false; /* are we debugging? */
-bool need_cleanup = false; /* do we need to remove files from
- * archive? */
-
-#ifndef WIN32
-static volatile sig_atomic_t signaled = false;
-#endif
-
-char *archiveLocation; /* where to find the archive? */
-char *triggerPath; /* where to find the trigger file? */
-char *xlogFilePath; /* where we are going to restore to */
-char *nextWALFileName; /* the file we need to get from archive */
-char *restartWALFileName; /* the file from which we can restart restore */
-char WALFilePath[MAXPGPATH * 2]; /* the file path including archive */
-char restoreCommand[MAXPGPATH]; /* run this to restore */
-char exclusiveCleanupFileName[MAXFNAMELEN]; /* the file we need to get
- * from archive */
-
-/*
- * Two types of failover are supported (smart and fast failover).
- *
- * The content of the trigger file determines the type of failover. If the
- * trigger file contains the word "smart" (or the file is empty), smart
- * failover is chosen: pg_standby acts as cp or ln command itself, on
- * successful completion all the available WAL records will be applied
- * resulting in zero data loss. But, it might take a long time to finish
- * recovery if there's a lot of unapplied WAL.
- *
- * On the other hand, if the trigger file contains the word "fast", the
- * recovery is finished immediately even if unapplied WAL files remain. Any
- * transactions in the unapplied WAL files are lost.
- *
- * An empty trigger file performs smart failover. SIGUSR or SIGINT triggers
- * fast failover. A timeout causes fast failover (smart failover would have
- * the same effect, since if the timeout is reached there is no unapplied WAL).
- */
-#define NoFailover 0
-#define SmartFailover 1
-#define FastFailover 2
-
-static int Failover = NoFailover;
-
-#define RESTORE_COMMAND_COPY 0
-#define RESTORE_COMMAND_LINK 1
-int restoreCommandType;
-
-#define XLOG_DATA 0
-#define XLOG_HISTORY 1
-int nextWALFileType;
-
-#define SET_RESTORE_COMMAND(cmd, arg1, arg2) \
- snprintf(restoreCommand, MAXPGPATH, cmd " \"%s\" \"%s\"", arg1, arg2)
-
-struct stat stat_buf;
-
-static bool SetWALFileNameForCleanup(void);
-static bool SetWALSegSize(void);
-
-
-/* =====================================================================
- *
- * Customizable section
- *
- * =====================================================================
- *
- * Currently, this section assumes that the Archive is a locally
- * accessible directory. If you want to make other assumptions,
- * such as using a vendor-specific archive and access API, these
- * routines are the ones you'll need to change. You're
- * encouraged to submit any changes to pgsql-hackers@lists.postgresql.org
- * or personally to the current maintainer. Those changes may be
- * folded in to later versions of this program.
- */
-
-/*
- * Initialize allows customized commands into the warm standby program.
- *
- * As an example, and probably the common case, we use either
- * cp/ln commands on *nix, or copy/move command on Windows.
- */
-static void
-CustomizableInitialize(void)
-{
-#ifdef WIN32
- snprintf(WALFilePath, MAXPGPATH, "%s\\%s", archiveLocation, nextWALFileName);
- switch (restoreCommandType)
- {
- case RESTORE_COMMAND_LINK:
- SET_RESTORE_COMMAND("mklink", WALFilePath, xlogFilePath);
- break;
- case RESTORE_COMMAND_COPY:
- default:
- SET_RESTORE_COMMAND("copy", WALFilePath, xlogFilePath);
- break;
- }
-#else
- snprintf(WALFilePath, MAXPGPATH, "%s/%s", archiveLocation, nextWALFileName);
- switch (restoreCommandType)
- {
- case RESTORE_COMMAND_LINK:
- SET_RESTORE_COMMAND("ln -s -f", WALFilePath, xlogFilePath);
- break;
- case RESTORE_COMMAND_COPY:
- default:
- SET_RESTORE_COMMAND("cp", WALFilePath, xlogFilePath);
- break;
- }
-#endif
-
- /*
- * This code assumes that archiveLocation is a directory You may wish to
- * add code to check for tape libraries, etc.. So, since it is a
- * directory, we use stat to test if it's accessible
- */
- if (stat(archiveLocation, &stat_buf) != 0)
- {
- fprintf(stderr, "%s: archive location \"%s\" does not exist\n", progname, archiveLocation);
- fflush(stderr);
- exit(2);
- }
-}
-
-/*
- * CustomizableNextWALFileReady()
- *
- * Is the requested file ready yet?
- */
-static bool
-CustomizableNextWALFileReady(void)
-{
- if (stat(WALFilePath, &stat_buf) == 0)
- {
- /*
- * If we've not seen any WAL segments, we don't know the WAL segment
- * size, which we need. If it looks like a WAL segment, determine size
- * of segments for the cluster.
- */
- if (WalSegSz == -1 && IsXLogFileName(nextWALFileName))
- {
- if (SetWALSegSize())
- {
- /*
- * Successfully determined WAL segment size. Can compute
- * cleanup cutoff now.
- */
- need_cleanup = SetWALFileNameForCleanup();
- if (debug)
- {
- fprintf(stderr,
- _("WAL segment size: %d \n"), WalSegSz);
- fprintf(stderr, "Keep archive history: ");
-
- if (need_cleanup)
- fprintf(stderr, "%s and later\n",
- exclusiveCleanupFileName);
- else
- fprintf(stderr, "no cleanup required\n");
- }
- }
- }
-
- /*
- * Return only if it's the right size already.
- */
- if (WalSegSz > 0 && stat_buf.st_size == WalSegSz)
- {
-#ifdef WIN32
-
- /*
- * Windows 'cp' sets the final file size before the copy is
- * complete, and not yet ready to be opened by pg_standby. So we
- * wait for sleeptime secs before attempting to restore. If that
- * is not enough, we will rely on the retry/holdoff mechanism.
- * GNUWin32's cp does not have this problem.
- */
- pg_usleep(sleeptime * 1000000L);
-#endif
- nextWALFileType = XLOG_DATA;
- return true;
- }
-
- /*
- * If still too small, wait until it is the correct size
- */
- if (WalSegSz > 0 && stat_buf.st_size > WalSegSz)
- {
- if (debug)
- {
- fprintf(stderr, "file size greater than expected\n");
- fflush(stderr);
- }
- exit(3);
- }
- }
-
- return false;
-}
-
-static void
-CustomizableCleanupPriorWALFiles(void)
-{
- /*
- * Work out name of prior file from current filename
- */
- if (nextWALFileType == XLOG_DATA)
- {
- int rc;
- DIR *xldir;
- struct dirent *xlde;
-
- /*
- * Assume it's OK to keep failing. The failure situation may change
- * over time, so we'd rather keep going on the main processing than
- * fail because we couldn't clean up yet.
- */
- if ((xldir = opendir(archiveLocation)) != NULL)
- {
- while (errno = 0, (xlde = readdir(xldir)) != NULL)
- {
- /*
- * We ignore the timeline part of the XLOG segment identifiers
- * in deciding whether a segment is still needed. This
- * ensures that we won't prematurely remove a segment from a
- * parent timeline. We could probably be a little more
- * proactive about removing segments of non-parent timelines,
- * but that would be a whole lot more complicated.
- *
- * We use the alphanumeric sorting property of the filenames
- * to decide which ones are earlier than the
- * exclusiveCleanupFileName file. Note that this means files
- * are not removed in the order they were originally written,
- * in case this worries you.
- */
- if (IsXLogFileName(xlde->d_name) &&
- strcmp(xlde->d_name + 8, exclusiveCleanupFileName + 8) < 0)
- {
-#ifdef WIN32
- snprintf(WALFilePath, sizeof(WALFilePath), "%s\\%s", archiveLocation, xlde->d_name);
-#else
- snprintf(WALFilePath, sizeof(WALFilePath), "%s/%s", archiveLocation, xlde->d_name);
-#endif
-
- if (debug)
- fprintf(stderr, "\nremoving file \"%s\"", WALFilePath);
-
- rc = unlink(WALFilePath);
- if (rc != 0)
- {
- fprintf(stderr, "\n%s: ERROR: could not remove file \"%s\": %s\n",
- progname, WALFilePath, strerror(errno));
- break;
- }
- }
- }
-
- if (errno)
- fprintf(stderr, "%s: could not read archive location \"%s\": %s\n",
- progname, archiveLocation, strerror(errno));
- if (debug)
- fprintf(stderr, "\n");
- }
- else
- fprintf(stderr, "%s: could not open archive location \"%s\": %s\n",
- progname, archiveLocation, strerror(errno));
-
- if (closedir(xldir))
- fprintf(stderr, "%s: could not close archive location \"%s\": %s\n",
- progname, archiveLocation, strerror(errno));
-
- fflush(stderr);
- }
-}
-
-/* =====================================================================
- * End of Customizable section
- * =====================================================================
- */
-
-/*
- * SetWALFileNameForCleanup()
- *
- * Set the earliest WAL filename that we want to keep on the archive
- * and decide whether we need_cleanup
- */
-static bool
-SetWALFileNameForCleanup(void)
-{
- uint32 tli = 1,
- log = 0,
- seg = 0;
- uint32 log_diff = 0,
- seg_diff = 0;
- bool cleanup = false;
- int max_segments_per_logfile = (0xFFFFFFFF / WalSegSz);
-
- if (restartWALFileName)
- {
- /*
- * Don't do cleanup if the restartWALFileName provided is later than
- * the xlog file requested. This is an error and we must not remove
- * these files from archive. This shouldn't happen, but better safe
- * than sorry.
- */
- if (strcmp(restartWALFileName, nextWALFileName) > 0)
- return false;
-
- strlcpy(exclusiveCleanupFileName, restartWALFileName, sizeof(exclusiveCleanupFileName));
- return true;
- }
-
- if (keepfiles > 0)
- {
- sscanf(nextWALFileName, "%08X%08X%08X", &tli, &log, &seg);
- if (tli > 0 && seg > 0)
- {
- log_diff = keepfiles / max_segments_per_logfile;
- seg_diff = keepfiles % max_segments_per_logfile;
- if (seg_diff > seg)
- {
- log_diff++;
- seg = max_segments_per_logfile - (seg_diff - seg);
- }
- else
- seg -= seg_diff;
-
- if (log >= log_diff)
- {
- log -= log_diff;
- cleanup = true;
- }
- else
- {
- log = 0;
- seg = 0;
- }
- }
- }
-
- XLogFileNameById(exclusiveCleanupFileName, tli, log, seg);
-
- return cleanup;
-}
-
-/*
- * Try to set the wal segment size from the WAL file specified by WALFilePath.
- *
- * Return true if size could be determined, false otherwise.
- */
-static bool
-SetWALSegSize(void)
-{
- bool ret_val = false;
- int fd;
- PGAlignedXLogBlock buf;
-
- Assert(WalSegSz == -1);
-
- if ((fd = open(WALFilePath, O_RDWR, 0)) < 0)
- {
- fprintf(stderr, "%s: could not open WAL file \"%s\": %s\n",
- progname, WALFilePath, strerror(errno));
- return false;
- }
-
- errno = 0;
- if (read(fd, buf.data, XLOG_BLCKSZ) == XLOG_BLCKSZ)
- {
- XLogLongPageHeader longhdr = (XLogLongPageHeader) buf.data;
-
- WalSegSz = longhdr->xlp_seg_size;
-
- if (IsValidWalSegSize(WalSegSz))
- {
- /* successfully retrieved WAL segment size */
- ret_val = true;
- }
- else
- fprintf(stderr,
- "%s: WAL segment size must be a power of two between 1MB and 1GB, but the WAL file header specifies %d bytes\n",
- progname, WalSegSz);
- }
- else
- {
- /*
- * Don't complain loudly, this is to be expected for segments being
- * created.
- */
- if (errno != 0)
- {
- if (debug)
- fprintf(stderr, "could not read file \"%s\": %s\n",
- WALFilePath, strerror(errno));
- }
- else
- {
- if (debug)
- fprintf(stderr, "not enough data in file \"%s\"\n",
- WALFilePath);
- }
- }
-
- fflush(stderr);
-
- close(fd);
- return ret_val;
-}
-
-/*
- * CheckForExternalTrigger()
- *
- * Is there a trigger file? Sets global 'Failover' variable to indicate
- * what kind of a trigger file it was. A "fast" trigger file is turned
- * into a "smart" file as a side-effect.
- */
-static void
-CheckForExternalTrigger(void)
-{
- char buf[32];
- int fd;
- int len;
-
- /*
- * Look for a trigger file, if that option has been selected
- *
- * We use stat() here because triggerPath is always a file rather than
- * potentially being in an archive
- */
- if (!triggerPath || stat(triggerPath, &stat_buf) != 0)
- return;
-
- /*
- * An empty trigger file performs smart failover. There's a little race
- * condition here: if the writer of the trigger file has just created the
- * file, but not yet written anything to it, we'll treat that as smart
- * shutdown even if the other process was just about to write "fast" to
- * it. But that's fine: we'll restore one more WAL file, and when we're
- * invoked next time, we'll see the word "fast" and fail over immediately.
- */
- if (stat_buf.st_size == 0)
- {
- Failover = SmartFailover;
- fprintf(stderr, "trigger file found: smart failover\n");
- fflush(stderr);
- return;
- }
-
- if ((fd = open(triggerPath, O_RDWR, 0)) < 0)
- {
- fprintf(stderr, "WARNING: could not open \"%s\": %s\n",
- triggerPath, strerror(errno));
- fflush(stderr);
- return;
- }
-
- if ((len = read(fd, buf, sizeof(buf) - 1)) < 0)
- {
- fprintf(stderr, "WARNING: could not read \"%s\": %s\n",
- triggerPath, strerror(errno));
- fflush(stderr);
- close(fd);
- return;
- }
- buf[len] = '\0';
-
- if (strncmp(buf, "smart", 5) == 0)
- {
- Failover = SmartFailover;
- fprintf(stderr, "trigger file found: smart failover\n");
- fflush(stderr);
- close(fd);
- return;
- }
-
- if (strncmp(buf, "fast", 4) == 0)
- {
- Failover = FastFailover;
-
- fprintf(stderr, "trigger file found: fast failover\n");
- fflush(stderr);
-
- /*
- * Turn it into a "smart" trigger by truncating the file. Otherwise if
- * the server asks us again to restore a segment that was restored
- * already, we would return "not found" and upset the server.
- */
- if (ftruncate(fd, 0) < 0)
- {
- fprintf(stderr, "WARNING: could not read \"%s\": %s\n",
- triggerPath, strerror(errno));
- fflush(stderr);
- }
- close(fd);
-
- return;
- }
- close(fd);
-
- fprintf(stderr, "WARNING: invalid content in \"%s\"\n", triggerPath);
- fflush(stderr);
-}
-
-/*
- * RestoreWALFileForRecovery()
- *
- * Perform the action required to restore the file from archive
- */
-static bool
-RestoreWALFileForRecovery(void)
-{
- int rc = 0;
- int numretries = 0;
-
- if (debug)
- {
- fprintf(stderr, "running restore: ");
- fflush(stderr);
- }
-
- while (numretries <= maxretries)
- {
- rc = system(restoreCommand);
- if (rc == 0)
- {
- if (debug)
- {
- fprintf(stderr, "OK\n");
- fflush(stderr);
- }
- return true;
- }
- pg_usleep(numretries++ * sleeptime * 1000000L);
- }
-
- /*
- * Allow caller to add additional info
- */
- if (debug)
- fprintf(stderr, "not restored\n");
- return false;
-}
-
-static void
-usage(void)
-{
- printf("%s allows PostgreSQL warm standby servers to be configured.\n\n", progname);
- printf("Usage:\n");
- printf(" %s [OPTION]... ARCHIVELOCATION NEXTWALFILE XLOGFILEPATH [RESTARTWALFILE]\n", progname);
- printf("\nOptions:\n");
- printf(" -c copy file from archive (default)\n");
- printf(" -d generate lots of debugging output (testing only)\n");
- printf(" -k NUMFILESTOKEEP if RESTARTWALFILE is not used, remove files prior to limit\n"
- " (0 keeps all)\n");
- printf(" -l does nothing; use of link is now deprecated\n");
- printf(" -r MAXRETRIES max number of times to retry, with progressive wait\n"
- " (default=3)\n");
- printf(" -s SLEEPTIME seconds to wait between file checks (min=1, max=60,\n"
- " default=5)\n");
- printf(" -t TRIGGERFILE trigger file to initiate failover (no default)\n");
- printf(" -V, --version output version information, then exit\n");
- printf(" -w MAXWAITTIME max seconds to wait for a file (0=no limit) (default=0)\n");
- printf(" -?, --help show this help, then exit\n");
- printf("\n"
- "Main intended use as restore_command in postgresql.conf:\n"
- " restore_command = 'pg_standby [OPTION]... ARCHIVELOCATION %%f %%p %%r'\n"
- "e.g.\n"
- " restore_command = 'pg_standby /mnt/server/archiverdir %%f %%p %%r'\n");
- printf("\nReport bugs to <%s>.\n", PACKAGE_BUGREPORT);
- printf("%s home page: <%s>\n", PACKAGE_NAME, PACKAGE_URL);
-}
-
-#ifndef WIN32
-static void
-sighandler(int sig)
-{
- signaled = true;
-}
-
-/* We don't want SIGQUIT to core dump */
-static void
-sigquit_handler(int sig)
-{
- pqsignal(SIGINT, SIG_DFL);
- kill(getpid(), SIGINT);
-}
-#endif
-
-/*------------ MAIN ----------------------------------------*/
-int
-main(int argc, char **argv)
-{
- int c;
-
- progname = get_progname(argv[0]);
-
- if (argc > 1)
- {
- if (strcmp(argv[1], "--help") == 0 || strcmp(argv[1], "-?") == 0)
- {
- usage();
- exit(0);
- }
- if (strcmp(argv[1], "--version") == 0 || strcmp(argv[1], "-V") == 0)
- {
- puts("pg_standby (PostgreSQL) " PG_VERSION);
- exit(0);
- }
- }
-
-#ifndef WIN32
-
- /*
- * You can send SIGUSR1 to trigger failover.
- *
- * Postmaster uses SIGQUIT to request immediate shutdown. The default
- * action is to core dump, but we don't want that, so trap it and commit
- * suicide without core dump.
- *
- * We used to use SIGINT and SIGQUIT to trigger failover, but that turned
- * out to be a bad idea because postmaster uses SIGQUIT to request
- * immediate shutdown. We still trap SIGINT, but that may change in a
- * future release.
- *
- * There's no way to trigger failover via signal on Windows.
- */
- (void) pqsignal(SIGUSR1, sighandler);
- (void) pqsignal(SIGINT, sighandler); /* deprecated, use SIGUSR1 */
- (void) pqsignal(SIGQUIT, sigquit_handler);
-#endif
-
- while ((c = getopt(argc, argv, "cdk:lr:s:t:w:")) != -1)
- {
- switch (c)
- {
- case 'c': /* Use copy */
- restoreCommandType = RESTORE_COMMAND_COPY;
- break;
- case 'd': /* Debug mode */
- debug = true;
- break;
- case 'k': /* keepfiles */
- keepfiles = atoi(optarg);
- if (keepfiles < 0)
- {
- fprintf(stderr, "%s: -k keepfiles must be >= 0\n", progname);
- exit(2);
- }
- break;
- case 'l': /* Use link */
-
- /*
- * Link feature disabled, possibly permanently. Linking causes
- * a problem after recovery ends that is not currently
- * resolved by PostgreSQL. 25 Jun 2009
- */
-#ifdef NOT_USED
- restoreCommandType = RESTORE_COMMAND_LINK;
-#endif
- break;
- case 'r': /* Retries */
- maxretries = atoi(optarg);
- if (maxretries < 0)
- {
- fprintf(stderr, "%s: -r maxretries must be >= 0\n", progname);
- exit(2);
- }
- break;
- case 's': /* Sleep time */
- sleeptime = atoi(optarg);
- if (sleeptime <= 0 || sleeptime > 60)
- {
- fprintf(stderr, "%s: -s sleeptime incorrectly set\n", progname);
- exit(2);
- }
- break;
- case 't': /* Trigger file */
- triggerPath = pg_strdup(optarg);
- break;
- case 'w': /* Max wait time */
- maxwaittime = atoi(optarg);
- if (maxwaittime < 0)
- {
- fprintf(stderr, "%s: -w maxwaittime incorrectly set\n", progname);
- exit(2);
- }
- break;
- default:
- fprintf(stderr, "Try \"%s --help\" for more information.\n", progname);
- exit(2);
- break;
- }
- }
-
- /*
- * Parameter checking - after checking to see if trigger file present
- */
- if (argc == 1)
- {
- fprintf(stderr, "%s: not enough command-line arguments\n", progname);
- exit(2);
- }
-
- /*
- * We will go to the archiveLocation to get nextWALFileName.
- * nextWALFileName may not exist yet, which would not be an error, so we
- * separate the archiveLocation and nextWALFileName so we can check
- * separately whether archiveLocation exists, if not that is an error
- */
- if (optind < argc)
- {
- archiveLocation = argv[optind];
- optind++;
- }
- else
- {
- fprintf(stderr, "%s: must specify archive location\n", progname);
- fprintf(stderr, "Try \"%s --help\" for more information.\n", progname);
- exit(2);
- }
-
- if (optind < argc)
- {
- nextWALFileName = argv[optind];
- optind++;
- }
- else
- {
- fprintf(stderr, "%s: must specify WAL file name as second non-option argument (use \"%%f\")\n", progname);
- fprintf(stderr, "Try \"%s --help\" for more information.\n", progname);
- exit(2);
- }
-
- if (optind < argc)
- {
- xlogFilePath = argv[optind];
- optind++;
- }
- else
- {
- fprintf(stderr, "%s: must specify xlog destination as third non-option argument (use \"%%p\")\n", progname);
- fprintf(stderr, "Try \"%s --help\" for more information.\n", progname);
- exit(2);
- }
-
- if (optind < argc)
- {
- restartWALFileName = argv[optind];
- optind++;
- }
-
- CustomizableInitialize();
-
- if (debug)
- {
- fprintf(stderr, "Trigger file: %s\n", triggerPath ? triggerPath : "<not set>");
- fprintf(stderr, "Waiting for WAL file: %s\n", nextWALFileName);
- fprintf(stderr, "WAL file path: %s\n", WALFilePath);
- fprintf(stderr, "Restoring to: %s\n", xlogFilePath);
- fprintf(stderr, "Sleep interval: %d second%s\n",
- sleeptime, (sleeptime > 1 ? "s" : " "));
- fprintf(stderr, "Max wait interval: %d %s\n",
- maxwaittime, (maxwaittime > 0 ? "seconds" : "forever"));
- fprintf(stderr, "Command for restore: %s\n", restoreCommand);
- fflush(stderr);
- }
-
- /*
- * Check for initial history file: always the first file to be requested
- * It's OK if the file isn't there - all other files need to wait
- */
- if (IsTLHistoryFileName(nextWALFileName))
- {
- nextWALFileType = XLOG_HISTORY;
- if (RestoreWALFileForRecovery())
- exit(0);
- else
- {
- if (debug)
- {
- fprintf(stderr, "history file not found\n");
- fflush(stderr);
- }
- exit(1);
- }
- }
-
- /*
- * Main wait loop
- */
- for (;;)
- {
- /* Check for trigger file or signal first */
- CheckForExternalTrigger();
-#ifndef WIN32
- if (signaled)
- {
- Failover = FastFailover;
- if (debug)
- {
- fprintf(stderr, "signaled to exit: fast failover\n");
- fflush(stderr);
- }
- }
-#endif
-
- /*
- * Check for fast failover immediately, before checking if the
- * requested WAL file is available
- */
- if (Failover == FastFailover)
- exit(1);
-
- if (CustomizableNextWALFileReady())
- {
- /*
- * Once we have restored this file successfully we can remove some
- * prior WAL files. If this restore fails we mustn't remove any
- * file because some of them will be requested again immediately
- * after the failed restore, or when we restart recovery.
- */
- if (RestoreWALFileForRecovery())
- {
- if (need_cleanup)
- CustomizableCleanupPriorWALFiles();
-
- exit(0);
- }
- else
- {
- /* Something went wrong in copying the file */
- exit(1);
- }
- }
-
- /* Check for smart failover if the next WAL file was not available */
- if (Failover == SmartFailover)
- exit(1);
-
- if (sleeptime <= 60)
- pg_usleep(sleeptime * 1000000L);
-
- waittime += sleeptime;
- if (waittime >= maxwaittime && maxwaittime > 0)
- {
- Failover = FastFailover;
- if (debug)
- {
- fprintf(stderr, "Timed out after %d seconds: fast failover\n",
- waittime);
- fflush(stderr);
- }
- }
- if (debug)
- {
- fprintf(stderr, "WAL file not present yet.");
- if (triggerPath)
- fprintf(stderr, " Checking for trigger file...");
- fprintf(stderr, "\n");
- fflush(stderr);
- }
- }
-}
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index ae2759be55..d3ca4b6932 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -192,13 +192,12 @@ pages.
<title>Server Applications</title>
<para>
- This section covers <productname>PostgreSQL</productname> server-related
- applications in <literal>contrib</literal>. They are typically run on the
- host where the database server resides. See also <xref
+ Some applications run on the <productname>PostgreSQL</productname> server
+ itself. Currently, no such applications are included in the
+ <literal>contrib</literal> directory. See also <xref
linkend="reference-server"/> for information about server applications that
are part of the core <productname>PostgreSQL</productname> distribution.
</para>
- &pgstandby;
</sect1>
</appendix>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 38e8aa0bbf..db1d369743 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -138,7 +138,6 @@
<!ENTITY pgfreespacemap SYSTEM "pgfreespacemap.sgml">
<!ENTITY pgprewarm SYSTEM "pgprewarm.sgml">
<!ENTITY pgrowlocks SYSTEM "pgrowlocks.sgml">
-<!ENTITY pgstandby SYSTEM "pgstandby.sgml">
<!ENTITY pgstatstatements SYSTEM "pgstatstatements.sgml">
<!ENTITY pgstattuple SYSTEM "pgstattuple.sgml">
<!ENTITY pgsurgery SYSTEM "pgsurgery.sgml">
diff --git a/doc/src/sgml/high-availability.sgml b/doc/src/sgml/high-availability.sgml
index 1892def655..b2ca63d370 100644
--- a/doc/src/sgml/high-availability.sgml
+++ b/doc/src/sgml/high-availability.sgml
@@ -702,8 +702,7 @@ protocol to make nodes agree on a serializable transactional order.
Do not use pg_standby or similar tools with the built-in standby mode
described here. <xref linkend="guc-restore-command"/> should return immediately
if the file does not exist; the server will retry the command again if
- necessary. See <xref linkend="log-shipping-alternative"/>
- for using tools like pg_standby.
+ necessary.
</para>
</note>
@@ -1482,8 +1481,7 @@ synchronous_standby_names = 'ANY 2 (s1, s2, s3)'
<para>
An alternative to the built-in standby mode described in the previous
sections is to use a <varname>restore_command</varname> that polls the archive location.
- This was the only option available in versions 8.4 and below. See the
- <xref linkend="pgstandby"/> module for a reference implementation of this.
+ This was the only option available in versions 8.4 and below.
</para>
<para>
@@ -1539,14 +1537,6 @@ if (!triggered)
</programlisting>
</para>
- <para>
- A working example of a waiting <varname>restore_command</varname> is provided
- in the <xref linkend="pgstandby"/> module. It
- should be used as a reference on how to correctly implement the logic
- described above. It can also be extended as needed to support specific
- configurations and environments.
- </para>
-
<para>
The method for triggering failover is an important part of planning
and design. One potential option is the <varname>restore_command</varname>
diff --git a/doc/src/sgml/pgstandby.sgml b/doc/src/sgml/pgstandby.sgml
deleted file mode 100644
index 66a6255930..0000000000
--- a/doc/src/sgml/pgstandby.sgml
+++ /dev/null
@@ -1,394 +0,0 @@
-<!-- doc/src/sgml/pgstandby.sgml -->
-
-<refentry id="pgstandby">
- <indexterm zone="pgstandby">
- <primary>pg_standby</primary>
- </indexterm>
-
- <refmeta>
- <refentrytitle><application>pg_standby</application></refentrytitle>
- <manvolnum>1</manvolnum>
- <refmiscinfo>Application</refmiscinfo>
- </refmeta>
-
- <refnamediv>
- <refname>pg_standby</refname>
- <refpurpose>supports the creation of a <productname>PostgreSQL</productname> warm standby server</refpurpose>
- </refnamediv>
-
- <refsynopsisdiv>
- <cmdsynopsis>
- <command>pg_standby</command>
- <arg rep="repeat"><replaceable>option</replaceable></arg>
- <arg choice="plain"><replaceable>archivelocation</replaceable></arg>
- <arg choice="plain"><replaceable>nextwalfile</replaceable></arg>
- <arg choice="plain"><replaceable>walfilepath</replaceable></arg>
- <arg choice="opt"><replaceable>restartwalfile</replaceable></arg>
- </cmdsynopsis>
- </refsynopsisdiv>
-
- <refsect1>
- <title>Description</title>
-
- <para>
- <application>pg_standby</application> supports creation of a <quote>warm standby</quote>
- database server. It is designed to be a production-ready program, as well
- as a customizable template should you require specific modifications.
- </para>
-
- <para>
- <application>pg_standby</application> is designed to be a waiting
- <varname>restore_command</varname>, which is needed to turn a standard
- archive recovery into a warm standby operation. Other
- configuration is required as well, all of which is described in the main
- server manual (see <xref linkend="warm-standby"/>).
- </para>
-
- <para>
- To configure a standby
- server to use <application>pg_standby</application>, put this into its
- <filename>postgresql.conf</filename> configuration file:
-<programlisting>
-restore_command = 'pg_standby <replaceable>archiveDir</replaceable> %f %p %r'
-</programlisting>
- where <replaceable>archiveDir</replaceable> is the directory from which WAL segment
- files should be restored.
- </para>
- <para>
- If <replaceable>restartwalfile</replaceable> is specified, normally by using the
- <literal>%r</literal> macro, then all WAL files logically preceding this
- file will be removed from <replaceable>archivelocation</replaceable>. This minimizes
- the number of files that need to be retained, while preserving
- crash-restart capability. Use of this parameter is appropriate if the
- <replaceable>archivelocation</replaceable> is a transient staging area for this
- particular standby server, but <emphasis>not</emphasis> when the
- <replaceable>archivelocation</replaceable> is intended as a long-term WAL archive area.
- </para>
- <para>
- <application>pg_standby</application> assumes that
- <replaceable>archivelocation</replaceable> is a directory readable by the
- server-owning user. If <replaceable>restartwalfile</replaceable> (or <literal>-k</literal>)
- is specified,
- the <replaceable>archivelocation</replaceable> directory must be writable too.
- </para>
- <para>
- There are two ways to fail over to a <quote>warm standby</quote> database server
- when the primary server fails:
-
- <variablelist>
- <varlistentry>
- <term>Smart Failover</term>
- <listitem>
- <para>
- In smart failover, the server is brought up after applying all WAL
- files available in the archive. This results in zero data loss, even if
- the standby server has fallen behind, but if there is a lot of
- unapplied WAL it can be a long time before the standby server becomes
- ready. To trigger a smart failover, create a trigger file containing
- the word <literal>smart</literal>, or just create it and leave it empty.
- </para>
- </listitem>
- </varlistentry>
- <varlistentry>
- <term>Fast Failover</term>
- <listitem>
- <para>
- In fast failover, the server is brought up immediately. Any WAL files
- in the archive that have not yet been applied will be ignored, and
- all transactions in those files are lost. To trigger a fast failover,
- create a trigger file and write the word <literal>fast</literal> into it.
- <application>pg_standby</application> can also be configured to execute a fast
- failover automatically if no new WAL file appears within a defined
- interval.
- </para>
- </listitem>
- </varlistentry>
- </variablelist>
- </para>
-
- </refsect1>
-
- <refsect1>
- <title>Options</title>
-
- <para>
- <application>pg_standby</application> accepts the following command-line arguments:
-
- <variablelist>
-
- <varlistentry>
- <term><option>-c</option></term>
- <listitem>
- <para>
- Use <literal>cp</literal> or <literal>copy</literal> command to restore WAL files
- from archive. This is the only supported behavior so this option is useless.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-d</option></term>
- <listitem>
- <para>
- Print lots of debug logging output on <filename>stderr</filename>.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-k</option></term>
- <listitem>
- <para>
- Remove files from <replaceable>archivelocation</replaceable> so that
- no more than this many WAL files before the current one are kept in the
- archive. Zero (the default) means not to remove any files from
- <replaceable>archivelocation</replaceable>.
- This parameter will be silently ignored if
- <replaceable>restartwalfile</replaceable> is specified, since that
- specification method is more accurate in determining the correct
- archive cut-off point.
- Use of this parameter is <emphasis>deprecated</emphasis> as of
- <productname>PostgreSQL</productname> 8.3; it is safer and more efficient to
- specify a <replaceable>restartwalfile</replaceable> parameter. A too
- small setting could result in removal of files that are still needed
- for a restart of the standby server, while a too large setting wastes
- archive space.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-r</option> <replaceable>maxretries</replaceable></term>
- <listitem>
- <para>
- Set the maximum number of times to retry the copy command if
- it fails (default 3). After each failure, we wait for
- <replaceable>sleeptime</replaceable> * <replaceable>num_retries</replaceable>
- so that the wait time increases progressively. So by default,
- we will wait 5 secs, 10 secs, then 15 secs before reporting
- the failure back to the standby server. This will be
- interpreted as end of recovery and the standby will come
- up fully as a result.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-s</option> <replaceable>sleeptime</replaceable></term>
- <listitem>
- <para>
- Set the number of seconds (up to 60, default 5) to sleep between
- tests to see if the WAL file to be restored is available in
- the archive yet. The default setting is not necessarily
- recommended; consult <xref linkend="warm-standby"/> for discussion.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-t</option> <replaceable>triggerfile</replaceable></term>
- <listitem>
- <para>
- Specify a trigger file whose presence should cause failover.
- It is recommended that you use a structured file name to
- avoid confusion as to which server is being triggered
- when multiple servers exist on the same system; for example
- <filename>/tmp/pgsql.trigger.5432</filename>.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-V</option></term>
- <term><option>--version</option></term>
- <listitem>
- <para>
- Print the <application>pg_standby</application> version and exit.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-w</option> <replaceable>maxwaittime</replaceable></term>
- <listitem>
- <para>
- Set the maximum number of seconds to wait for the next WAL file,
- after which a fast failover will be performed.
- A setting of zero (the default) means wait forever.
- The default setting is not necessarily recommended;
- consult <xref linkend="warm-standby"/> for discussion.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-?</option></term>
- <term><option>--help</option></term>
- <listitem>
- <para>
- Show help about <application>pg_standby</application> command line
- arguments, and exit.
- </para>
- </listitem>
- </varlistentry>
- </variablelist>
- </para>
-
- </refsect1>
-
- <refsect1>
- <title>Notes</title>
-
- <para>
- <application>pg_standby</application> is designed to work with
- <productname>PostgreSQL</productname> 8.2 and later.
- </para>
- <para>
- <productname>PostgreSQL</productname> 8.3 provides the <literal>%r</literal> macro,
- which is designed to let <application>pg_standby</application> know the
- last file it needs to keep. With <productname>PostgreSQL</productname> 8.2, the
- <literal>-k</literal> option must be used if archive cleanup is
- required. This option remains available in 8.3, but its use is deprecated.
- </para>
- <para>
- <productname>PostgreSQL</productname> 8.4 provides the
- <varname>recovery_end_command</varname> option. Without this option
- a leftover trigger file can be hazardous.
- </para>
-
- <para>
- <application>pg_standby</application> is written in C and has an
- easy-to-modify source code, with specifically designated sections to modify
- for your own needs
- </para>
- </refsect1>
-
- <refsect1>
- <title>Examples</title>
-
- <para>On Linux or Unix systems, you might use:
-
-<programlisting>
-archive_command = 'cp %p .../archive/%f'
-
-restore_command = 'pg_standby -d -s 2 -t /tmp/pgsql.trigger.5442 .../archive %f %p %r 2>>standby.log'
-
-recovery_end_command = 'rm -f /tmp/pgsql.trigger.5442'
-</programlisting>
- where the archive directory is physically located on the standby server,
- so that the <varname>archive_command</varname> is accessing it across NFS,
- but the files are local to the standby (enabling use of <literal>ln</literal>).
- This will:
- <itemizedlist>
- <listitem>
- <para>
- produce debugging output in <filename>standby.log</filename>
- </para>
- </listitem>
- <listitem>
- <para>
- sleep for 2 seconds between checks for next WAL file availability
- </para>
- </listitem>
- <listitem>
- <para>
- stop waiting only when a trigger file called
- <filename>/tmp/pgsql.trigger.5442</filename> appears,
- and perform failover according to its content
- </para>
- </listitem>
- <listitem>
- <para>
- remove the trigger file when recovery ends
- </para>
- </listitem>
- <listitem>
- <para>
- remove no-longer-needed files from the archive directory
- </para>
- </listitem>
- </itemizedlist>
- </para>
-
- <para>On Windows, you might use:
-
-<programlisting>
-archive_command = 'copy %p ...\\archive\\%f'
-
-restore_command = 'pg_standby -d -s 5 -t C:\pgsql.trigger.5442 ...\archive %f %p %r 2>>standby.log'
-
-recovery_end_command = 'del C:\pgsql.trigger.5442'
-</programlisting>
- Note that backslashes need to be doubled in the
- <varname>archive_command</varname>, but <emphasis>not</emphasis> in the
- <varname>restore_command</varname> or <varname>recovery_end_command</varname>.
- This will:
- <itemizedlist>
- <listitem>
- <para>
- use the <literal>copy</literal> command to restore WAL files from archive
- </para>
- </listitem>
- <listitem>
- <para>
- produce debugging output in <filename>standby.log</filename>
- </para>
- </listitem>
- <listitem>
- <para>
- sleep for 5 seconds between checks for next WAL file availability
- </para>
- </listitem>
- <listitem>
- <para>
- stop waiting only when a trigger file called
- <filename>C:\pgsql.trigger.5442</filename> appears,
- and perform failover according to its content
- </para>
- </listitem>
- <listitem>
- <para>
- remove the trigger file when recovery ends
- </para>
- </listitem>
- <listitem>
- <para>
- remove no-longer-needed files from the archive directory
- </para>
- </listitem>
- </itemizedlist>
- </para>
-
- <para>
- The <literal>copy</literal> command on Windows sets the final file size
- before the file is completely copied, which would ordinarily confuse
- <application>pg_standby</application>. Therefore
- <application>pg_standby</application> waits <replaceable>sleeptime</replaceable>
- seconds once it sees the proper file size. GNUWin32's <literal>cp</literal>
- sets the file size only after the file copy is complete.
- </para>
-
- <para>
- Since the Windows example uses <literal>copy</literal> at both ends, either
- or both servers might be accessing the archive directory across the
- network.
- </para>
-
- </refsect1>
-
- <refsect1>
- <title>Author</title>
-
- <para>
- Simon Riggs <email>simon@2ndquadrant.com</email>
- </para>
- </refsect1>
-
- <refsect1>
- <title>See Also</title>
-
- <simplelist type="inline">
- <member><xref linkend="pgarchivecleanup"/></member>
- </simplelist>
- </refsect1>
-</refentry>
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index c86323a6df..a0a6512b11 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -34,8 +34,8 @@ my @unlink_on_exit;
# Set of variables for modules in contrib/ and src/test/modules/
my $contrib_defines = { 'refint' => 'REFINT_VERBOSE' };
my @contrib_uselibpq = ('dblink', 'oid2name', 'postgres_fdw', 'vacuumlo');
-my @contrib_uselibpgport = ('oid2name', 'pg_standby', 'vacuumlo');
-my @contrib_uselibpgcommon = ('oid2name', 'pg_standby', 'vacuumlo');
+my @contrib_uselibpgport = ('oid2name', 'vacuumlo');
+my @contrib_uselibpgcommon = ('oid2name', 'vacuumlo');
my $contrib_extralibs = undef;
my $contrib_extraincludes = { 'dblink' => ['src/backend'] };
my $contrib_extrasource = {
--
2.17.0
* Re: [PATCH] remove pg_standby
2020-11-21 19:41 Re: [PATCH] remove pg_standby Justin Pryzby <pryzby@telsasoft.com>
@ 2020-11-25 09:04 ` Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2021-01-27 03:13 ` Re: [PATCH] remove pg_standby Thomas Munro <thomas.munro@gmail.com>
0 siblings, 1 reply; 25+ messages in thread
From: Peter Eisentraut @ 2020-11-25 09:04 UTC (permalink / raw)
To: Justin Pryzby <pryzby@telsasoft.com>; +Cc: Heikki Linnakangas <hlinnaka@iki.fi>; Tom Lane <tgl@sss.pgh.pa.us>; Stephen Frost <sfrost@snowman.net>; Ian Lawrence Barwick <barwick@gmail.com>; pgsql-hackers
On 2020-11-21 20:41, Justin Pryzby wrote:
> On Fri, Nov 20, 2020 at 05:26:54PM +0100, Peter Eisentraut wrote:
>> On 2020-10-29 03:44, Justin Pryzby wrote:
>>> diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
>>> index 4e833d79ef..be4292ec33 100644
>>> --- a/doc/src/sgml/contrib.sgml
>>> +++ b/doc/src/sgml/contrib.sgml
>>> @@ -199,6 +199,5 @@ pages.
>>> part of the core <productname>PostgreSQL</productname> distribution.
>>> </para>
>>> - &pgstandby;
>>> </sect1>
>>> </appendix>
>>
>> With this removal, that section becomes empty. So you probably want to
>> clean up or reorganize this a bit.
>>
>> See https://www.postgresql.org/docs/devel/contrib-prog.html for the context.
>
> Oops. I guess I'd write something like this. If we just remove it, then
> there'd no place to add a new server application, and "client applications"
> would be the only subsection.
I have committed the typo fix. I don't have a well-formed opinion yet
about whether all the reservations about removing pg_standby have been
addressed.
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: [PATCH] remove pg_standby
2020-11-21 19:41 Re: [PATCH] remove pg_standby Justin Pryzby <pryzby@telsasoft.com>
2020-11-25 09:04 ` Re: [PATCH] remove pg_standby Peter Eisentraut <peter.eisentraut@enterprisedb.com>
@ 2021-01-27 03:13 ` Thomas Munro <thomas.munro@gmail.com>
2021-01-27 05:06 ` Re: [PATCH] remove pg_standby Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 25+ messages in thread
From: Thomas Munro @ 2021-01-27 03:13 UTC (permalink / raw)
To: Peter Eisentraut <peter.eisentraut@enterprisedb.com>; +Cc: Justin Pryzby <pryzby@telsasoft.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Tom Lane <tgl@sss.pgh.pa.us>; Stephen Frost <sfrost@snowman.net>; Ian Lawrence Barwick <barwick@gmail.com>; pgsql-hackers
On Wed, Nov 25, 2020 at 10:04 PM Peter Eisentraut
<peter.eisentraut@enterprisedb.com> wrote:
> On 2020-11-21 20:41, Justin Pryzby wrote:
> > Oops. I guess I'd write something like this. If we just remove it, then
> > there'd no place to add a new server application, and "client applications"
> > would be the only subsection.
>
> I have committed the typo fix. I don't have a well-formed opinion yet
> about whether all the reservations about removing pg_standby have been
> addressed.
I would like to commit this, because "waiting restore commands" have
confusing interactions with my proposed prefetching-during-recovery
patch[1]. Here's a version that fixes an error when building the docs
(there was a stray remaining <xref linkend="pgstandby"/>), and adds a
commit message. Any objections?
Furthermore, I think we should also remove the section of the manual
that describes how to write your own "waiting restore command".
Thoughts?
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGKFeYPL9K%2BSRixcsx1%2B6HsHhqK%2BPOZyrnnZjw1jERpG...
Attachments: download all (2 files, .tar.gz)
v3-0001-Retire-pg_standby.patch (text/x-patch, 44.7K) download
From 54b12290e304bf1d4c995836addb7a53e8e7b74b Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Mon, 26 Oct 2020 16:37:46 -0500
Subject: [PATCH v3] Retire pg_standby.
Streaming replication made pg_standby obsolete. It has been proposed
that we retire it many times. Now seems like a good time to do it,
because "waiting restore commands" are incompatible with a proposed WAL
prefetching feature.
Discussion: https://postgr.es/m/20201029024412.GP5380%40telsasoft.com
Author: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
---
contrib/Makefile | 1 -
contrib/pg_standby/.gitignore | 1 -
contrib/pg_standby/Makefile | 20 -
contrib/pg_standby/pg_standby.c | 907 -------------------------
doc/src/sgml/contrib.sgml | 7 +-
doc/src/sgml/filelist.sgml | 1 -
doc/src/sgml/high-availability.sgml | 14 +-
doc/src/sgml/pgstandby.sgml | 394 -----------
doc/src/sgml/ref/pgarchivecleanup.sgml | 7 -
src/tools/msvc/Mkvcbuild.pm | 4 +-
10 files changed, 7 insertions(+), 1349 deletions(-)
delete mode 100644 contrib/pg_standby/.gitignore
delete mode 100644 contrib/pg_standby/Makefile
delete mode 100644 contrib/pg_standby/pg_standby.c
delete mode 100644 doc/src/sgml/pgstandby.sgml
diff --git a/contrib/Makefile b/contrib/Makefile
index 7a4866e338..cdc041c7db 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -33,7 +33,6 @@ SUBDIRS = \
pg_buffercache \
pg_freespacemap \
pg_prewarm \
- pg_standby \
pg_stat_statements \
pg_surgery \
pg_trgm \
diff --git a/contrib/pg_standby/.gitignore b/contrib/pg_standby/.gitignore
deleted file mode 100644
index a401b085a8..0000000000
--- a/contrib/pg_standby/.gitignore
+++ /dev/null
@@ -1 +0,0 @@
-/pg_standby
diff --git a/contrib/pg_standby/Makefile b/contrib/pg_standby/Makefile
deleted file mode 100644
index 87732bedf1..0000000000
--- a/contrib/pg_standby/Makefile
+++ /dev/null
@@ -1,20 +0,0 @@
-# contrib/pg_standby/Makefile
-
-PGFILEDESC = "pg_standby - supports creation of a warm standby"
-PGAPPICON = win32
-
-PROGRAM = pg_standby
-OBJS = \
- $(WIN32RES) \
- pg_standby.o
-
-ifdef USE_PGXS
-PG_CONFIG = pg_config
-PGXS := $(shell $(PG_CONFIG) --pgxs)
-include $(PGXS)
-else
-subdir = contrib/pg_standby
-top_builddir = ../..
-include $(top_builddir)/src/Makefile.global
-include $(top_srcdir)/contrib/contrib-global.mk
-endif
diff --git a/contrib/pg_standby/pg_standby.c b/contrib/pg_standby/pg_standby.c
deleted file mode 100644
index c9f33e4254..0000000000
--- a/contrib/pg_standby/pg_standby.c
+++ /dev/null
@@ -1,907 +0,0 @@
-/*
- * contrib/pg_standby/pg_standby.c
- *
- *
- * pg_standby.c
- *
- * Production-ready example of how to create a Warm Standby
- * database server using continuous archiving as a
- * replication mechanism
- *
- * We separate the parameters for archive and nextWALfile
- * so that we can check the archive exists, even if the
- * WAL file doesn't (yet).
- *
- * This program will be executed once in full for each file
- * requested by the warm standby server.
- *
- * It is designed to cater to a variety of needs, as well
- * providing a customizable section.
- *
- * Original author: Simon Riggs simon@2ndquadrant.com
- * Current maintainer: Simon Riggs
- */
-#include "postgres_fe.h"
-
-#include <ctype.h>
-#include <dirent.h>
-#include <sys/stat.h>
-#include <fcntl.h>
-#include <signal.h>
-#include <sys/time.h>
-
-#include "access/xlog_internal.h"
-#include "pg_getopt.h"
-
-const char *progname;
-
-int WalSegSz = -1;
-
-/* Options and defaults */
-int sleeptime = 5; /* amount of time to sleep between file checks */
-int waittime = -1; /* how long we have been waiting, -1 no wait
- * yet */
-int maxwaittime = 0; /* how long are we prepared to wait for? */
-int keepfiles = 0; /* number of WAL files to keep, 0 keep all */
-int maxretries = 3; /* number of retries on restore command */
-bool debug = false; /* are we debugging? */
-bool need_cleanup = false; /* do we need to remove files from
- * archive? */
-
-#ifndef WIN32
-static volatile sig_atomic_t signaled = false;
-#endif
-
-char *archiveLocation; /* where to find the archive? */
-char *triggerPath; /* where to find the trigger file? */
-char *xlogFilePath; /* where we are going to restore to */
-char *nextWALFileName; /* the file we need to get from archive */
-char *restartWALFileName; /* the file from which we can restart restore */
-char WALFilePath[MAXPGPATH * 2]; /* the file path including archive */
-char restoreCommand[MAXPGPATH]; /* run this to restore */
-char exclusiveCleanupFileName[MAXFNAMELEN]; /* the file we need to get
- * from archive */
-
-/*
- * Two types of failover are supported (smart and fast failover).
- *
- * The content of the trigger file determines the type of failover. If the
- * trigger file contains the word "smart" (or the file is empty), smart
- * failover is chosen: pg_standby acts as cp or ln command itself, on
- * successful completion all the available WAL records will be applied
- * resulting in zero data loss. But, it might take a long time to finish
- * recovery if there's a lot of unapplied WAL.
- *
- * On the other hand, if the trigger file contains the word "fast", the
- * recovery is finished immediately even if unapplied WAL files remain. Any
- * transactions in the unapplied WAL files are lost.
- *
- * An empty trigger file performs smart failover. SIGUSR or SIGINT triggers
- * fast failover. A timeout causes fast failover (smart failover would have
- * the same effect, since if the timeout is reached there is no unapplied WAL).
- */
-#define NoFailover 0
-#define SmartFailover 1
-#define FastFailover 2
-
-static int Failover = NoFailover;
-
-#define RESTORE_COMMAND_COPY 0
-#define RESTORE_COMMAND_LINK 1
-int restoreCommandType;
-
-#define XLOG_DATA 0
-#define XLOG_HISTORY 1
-int nextWALFileType;
-
-#define SET_RESTORE_COMMAND(cmd, arg1, arg2) \
- snprintf(restoreCommand, MAXPGPATH, cmd " \"%s\" \"%s\"", arg1, arg2)
-
-struct stat stat_buf;
-
-static bool SetWALFileNameForCleanup(void);
-static bool SetWALSegSize(void);
-
-
-/* =====================================================================
- *
- * Customizable section
- *
- * =====================================================================
- *
- * Currently, this section assumes that the Archive is a locally
- * accessible directory. If you want to make other assumptions,
- * such as using a vendor-specific archive and access API, these
- * routines are the ones you'll need to change. You're
- * encouraged to submit any changes to pgsql-hackers@lists.postgresql.org
- * or personally to the current maintainer. Those changes may be
- * folded in to later versions of this program.
- */
-
-/*
- * Initialize allows customized commands into the warm standby program.
- *
- * As an example, and probably the common case, we use either
- * cp/ln commands on *nix, or copy/move command on Windows.
- */
-static void
-CustomizableInitialize(void)
-{
-#ifdef WIN32
- snprintf(WALFilePath, MAXPGPATH, "%s\\%s", archiveLocation, nextWALFileName);
- switch (restoreCommandType)
- {
- case RESTORE_COMMAND_LINK:
- SET_RESTORE_COMMAND("mklink", WALFilePath, xlogFilePath);
- break;
- case RESTORE_COMMAND_COPY:
- default:
- SET_RESTORE_COMMAND("copy", WALFilePath, xlogFilePath);
- break;
- }
-#else
- snprintf(WALFilePath, MAXPGPATH, "%s/%s", archiveLocation, nextWALFileName);
- switch (restoreCommandType)
- {
- case RESTORE_COMMAND_LINK:
- SET_RESTORE_COMMAND("ln -s -f", WALFilePath, xlogFilePath);
- break;
- case RESTORE_COMMAND_COPY:
- default:
- SET_RESTORE_COMMAND("cp", WALFilePath, xlogFilePath);
- break;
- }
-#endif
-
- /*
- * This code assumes that archiveLocation is a directory You may wish to
- * add code to check for tape libraries, etc.. So, since it is a
- * directory, we use stat to test if it's accessible
- */
- if (stat(archiveLocation, &stat_buf) != 0)
- {
- fprintf(stderr, "%s: archive location \"%s\" does not exist\n", progname, archiveLocation);
- fflush(stderr);
- exit(2);
- }
-}
-
-/*
- * CustomizableNextWALFileReady()
- *
- * Is the requested file ready yet?
- */
-static bool
-CustomizableNextWALFileReady(void)
-{
- if (stat(WALFilePath, &stat_buf) == 0)
- {
- /*
- * If we've not seen any WAL segments, we don't know the WAL segment
- * size, which we need. If it looks like a WAL segment, determine size
- * of segments for the cluster.
- */
- if (WalSegSz == -1 && IsXLogFileName(nextWALFileName))
- {
- if (SetWALSegSize())
- {
- /*
- * Successfully determined WAL segment size. Can compute
- * cleanup cutoff now.
- */
- need_cleanup = SetWALFileNameForCleanup();
- if (debug)
- {
- fprintf(stderr,
- _("WAL segment size: %d \n"), WalSegSz);
- fprintf(stderr, "Keep archive history: ");
-
- if (need_cleanup)
- fprintf(stderr, "%s and later\n",
- exclusiveCleanupFileName);
- else
- fprintf(stderr, "no cleanup required\n");
- }
- }
- }
-
- /*
- * Return only if it's the right size already.
- */
- if (WalSegSz > 0 && stat_buf.st_size == WalSegSz)
- {
-#ifdef WIN32
-
- /*
- * Windows 'cp' sets the final file size before the copy is
- * complete, and not yet ready to be opened by pg_standby. So we
- * wait for sleeptime secs before attempting to restore. If that
- * is not enough, we will rely on the retry/holdoff mechanism.
- * GNUWin32's cp does not have this problem.
- */
- pg_usleep(sleeptime * 1000000L);
-#endif
- nextWALFileType = XLOG_DATA;
- return true;
- }
-
- /*
- * If still too small, wait until it is the correct size
- */
- if (WalSegSz > 0 && stat_buf.st_size > WalSegSz)
- {
- if (debug)
- {
- fprintf(stderr, "file size greater than expected\n");
- fflush(stderr);
- }
- exit(3);
- }
- }
-
- return false;
-}
-
-static void
-CustomizableCleanupPriorWALFiles(void)
-{
- /*
- * Work out name of prior file from current filename
- */
- if (nextWALFileType == XLOG_DATA)
- {
- int rc;
- DIR *xldir;
- struct dirent *xlde;
-
- /*
- * Assume it's OK to keep failing. The failure situation may change
- * over time, so we'd rather keep going on the main processing than
- * fail because we couldn't clean up yet.
- */
- if ((xldir = opendir(archiveLocation)) != NULL)
- {
- while (errno = 0, (xlde = readdir(xldir)) != NULL)
- {
- /*
- * We ignore the timeline part of the XLOG segment identifiers
- * in deciding whether a segment is still needed. This
- * ensures that we won't prematurely remove a segment from a
- * parent timeline. We could probably be a little more
- * proactive about removing segments of non-parent timelines,
- * but that would be a whole lot more complicated.
- *
- * We use the alphanumeric sorting property of the filenames
- * to decide which ones are earlier than the
- * exclusiveCleanupFileName file. Note that this means files
- * are not removed in the order they were originally written,
- * in case this worries you.
- */
- if (IsXLogFileName(xlde->d_name) &&
- strcmp(xlde->d_name + 8, exclusiveCleanupFileName + 8) < 0)
- {
-#ifdef WIN32
- snprintf(WALFilePath, sizeof(WALFilePath), "%s\\%s", archiveLocation, xlde->d_name);
-#else
- snprintf(WALFilePath, sizeof(WALFilePath), "%s/%s", archiveLocation, xlde->d_name);
-#endif
-
- if (debug)
- fprintf(stderr, "\nremoving file \"%s\"", WALFilePath);
-
- rc = unlink(WALFilePath);
- if (rc != 0)
- {
- fprintf(stderr, "\n%s: ERROR: could not remove file \"%s\": %s\n",
- progname, WALFilePath, strerror(errno));
- break;
- }
- }
- }
-
- if (errno)
- fprintf(stderr, "%s: could not read archive location \"%s\": %s\n",
- progname, archiveLocation, strerror(errno));
- if (debug)
- fprintf(stderr, "\n");
- }
- else
- fprintf(stderr, "%s: could not open archive location \"%s\": %s\n",
- progname, archiveLocation, strerror(errno));
-
- if (closedir(xldir))
- fprintf(stderr, "%s: could not close archive location \"%s\": %s\n",
- progname, archiveLocation, strerror(errno));
-
- fflush(stderr);
- }
-}
-
-/* =====================================================================
- * End of Customizable section
- * =====================================================================
- */
-
-/*
- * SetWALFileNameForCleanup()
- *
- * Set the earliest WAL filename that we want to keep on the archive
- * and decide whether we need_cleanup
- */
-static bool
-SetWALFileNameForCleanup(void)
-{
- uint32 tli = 1,
- log = 0,
- seg = 0;
- uint32 log_diff = 0,
- seg_diff = 0;
- bool cleanup = false;
- int max_segments_per_logfile = (0xFFFFFFFF / WalSegSz);
-
- if (restartWALFileName)
- {
- /*
- * Don't do cleanup if the restartWALFileName provided is later than
- * the xlog file requested. This is an error and we must not remove
- * these files from archive. This shouldn't happen, but better safe
- * than sorry.
- */
- if (strcmp(restartWALFileName, nextWALFileName) > 0)
- return false;
-
- strlcpy(exclusiveCleanupFileName, restartWALFileName, sizeof(exclusiveCleanupFileName));
- return true;
- }
-
- if (keepfiles > 0)
- {
- sscanf(nextWALFileName, "%08X%08X%08X", &tli, &log, &seg);
- if (tli > 0 && seg > 0)
- {
- log_diff = keepfiles / max_segments_per_logfile;
- seg_diff = keepfiles % max_segments_per_logfile;
- if (seg_diff > seg)
- {
- log_diff++;
- seg = max_segments_per_logfile - (seg_diff - seg);
- }
- else
- seg -= seg_diff;
-
- if (log >= log_diff)
- {
- log -= log_diff;
- cleanup = true;
- }
- else
- {
- log = 0;
- seg = 0;
- }
- }
- }
-
- XLogFileNameById(exclusiveCleanupFileName, tli, log, seg);
-
- return cleanup;
-}
-
-/*
- * Try to set the wal segment size from the WAL file specified by WALFilePath.
- *
- * Return true if size could be determined, false otherwise.
- */
-static bool
-SetWALSegSize(void)
-{
- bool ret_val = false;
- int fd;
- PGAlignedXLogBlock buf;
-
- Assert(WalSegSz == -1);
-
- if ((fd = open(WALFilePath, O_RDWR, 0)) < 0)
- {
- fprintf(stderr, "%s: could not open WAL file \"%s\": %s\n",
- progname, WALFilePath, strerror(errno));
- return false;
- }
-
- errno = 0;
- if (read(fd, buf.data, XLOG_BLCKSZ) == XLOG_BLCKSZ)
- {
- XLogLongPageHeader longhdr = (XLogLongPageHeader) buf.data;
-
- WalSegSz = longhdr->xlp_seg_size;
-
- if (IsValidWalSegSize(WalSegSz))
- {
- /* successfully retrieved WAL segment size */
- ret_val = true;
- }
- else
- fprintf(stderr,
- "%s: WAL segment size must be a power of two between 1MB and 1GB, but the WAL file header specifies %d bytes\n",
- progname, WalSegSz);
- }
- else
- {
- /*
- * Don't complain loudly, this is to be expected for segments being
- * created.
- */
- if (errno != 0)
- {
- if (debug)
- fprintf(stderr, "could not read file \"%s\": %s\n",
- WALFilePath, strerror(errno));
- }
- else
- {
- if (debug)
- fprintf(stderr, "not enough data in file \"%s\"\n",
- WALFilePath);
- }
- }
-
- fflush(stderr);
-
- close(fd);
- return ret_val;
-}
-
-/*
- * CheckForExternalTrigger()
- *
- * Is there a trigger file? Sets global 'Failover' variable to indicate
- * what kind of a trigger file it was. A "fast" trigger file is turned
- * into a "smart" file as a side-effect.
- */
-static void
-CheckForExternalTrigger(void)
-{
- char buf[32];
- int fd;
- int len;
-
- /*
- * Look for a trigger file, if that option has been selected
- *
- * We use stat() here because triggerPath is always a file rather than
- * potentially being in an archive
- */
- if (!triggerPath || stat(triggerPath, &stat_buf) != 0)
- return;
-
- /*
- * An empty trigger file performs smart failover. There's a little race
- * condition here: if the writer of the trigger file has just created the
- * file, but not yet written anything to it, we'll treat that as smart
- * shutdown even if the other process was just about to write "fast" to
- * it. But that's fine: we'll restore one more WAL file, and when we're
- * invoked next time, we'll see the word "fast" and fail over immediately.
- */
- if (stat_buf.st_size == 0)
- {
- Failover = SmartFailover;
- fprintf(stderr, "trigger file found: smart failover\n");
- fflush(stderr);
- return;
- }
-
- if ((fd = open(triggerPath, O_RDWR, 0)) < 0)
- {
- fprintf(stderr, "WARNING: could not open \"%s\": %s\n",
- triggerPath, strerror(errno));
- fflush(stderr);
- return;
- }
-
- if ((len = read(fd, buf, sizeof(buf) - 1)) < 0)
- {
- fprintf(stderr, "WARNING: could not read \"%s\": %s\n",
- triggerPath, strerror(errno));
- fflush(stderr);
- close(fd);
- return;
- }
- buf[len] = '\0';
-
- if (strncmp(buf, "smart", 5) == 0)
- {
- Failover = SmartFailover;
- fprintf(stderr, "trigger file found: smart failover\n");
- fflush(stderr);
- close(fd);
- return;
- }
-
- if (strncmp(buf, "fast", 4) == 0)
- {
- Failover = FastFailover;
-
- fprintf(stderr, "trigger file found: fast failover\n");
- fflush(stderr);
-
- /*
- * Turn it into a "smart" trigger by truncating the file. Otherwise if
- * the server asks us again to restore a segment that was restored
- * already, we would return "not found" and upset the server.
- */
- if (ftruncate(fd, 0) < 0)
- {
- fprintf(stderr, "WARNING: could not read \"%s\": %s\n",
- triggerPath, strerror(errno));
- fflush(stderr);
- }
- close(fd);
-
- return;
- }
- close(fd);
-
- fprintf(stderr, "WARNING: invalid content in \"%s\"\n", triggerPath);
- fflush(stderr);
-}
-
-/*
- * RestoreWALFileForRecovery()
- *
- * Perform the action required to restore the file from archive
- */
-static bool
-RestoreWALFileForRecovery(void)
-{
- int rc = 0;
- int numretries = 0;
-
- if (debug)
- {
- fprintf(stderr, "running restore: ");
- fflush(stderr);
- }
-
- while (numretries <= maxretries)
- {
- rc = system(restoreCommand);
- if (rc == 0)
- {
- if (debug)
- {
- fprintf(stderr, "OK\n");
- fflush(stderr);
- }
- return true;
- }
- pg_usleep(numretries++ * sleeptime * 1000000L);
- }
-
- /*
- * Allow caller to add additional info
- */
- if (debug)
- fprintf(stderr, "not restored\n");
- return false;
-}
-
-static void
-usage(void)
-{
- printf("%s allows PostgreSQL warm standby servers to be configured.\n\n", progname);
- printf("Usage:\n");
- printf(" %s [OPTION]... ARCHIVELOCATION NEXTWALFILE XLOGFILEPATH [RESTARTWALFILE]\n", progname);
- printf("\nOptions:\n");
- printf(" -c copy file from archive (default)\n");
- printf(" -d generate lots of debugging output (testing only)\n");
- printf(" -k NUMFILESTOKEEP if RESTARTWALFILE is not used, remove files prior to limit\n"
- " (0 keeps all)\n");
- printf(" -l does nothing; use of link is now deprecated\n");
- printf(" -r MAXRETRIES max number of times to retry, with progressive wait\n"
- " (default=3)\n");
- printf(" -s SLEEPTIME seconds to wait between file checks (min=1, max=60,\n"
- " default=5)\n");
- printf(" -t TRIGGERFILE trigger file to initiate failover (no default)\n");
- printf(" -V, --version output version information, then exit\n");
- printf(" -w MAXWAITTIME max seconds to wait for a file (0=no limit) (default=0)\n");
- printf(" -?, --help show this help, then exit\n");
- printf("\n"
- "Main intended use as restore_command in postgresql.conf:\n"
- " restore_command = 'pg_standby [OPTION]... ARCHIVELOCATION %%f %%p %%r'\n"
- "e.g.\n"
- " restore_command = 'pg_standby /mnt/server/archiverdir %%f %%p %%r'\n");
- printf("\nReport bugs to <%s>.\n", PACKAGE_BUGREPORT);
- printf("%s home page: <%s>\n", PACKAGE_NAME, PACKAGE_URL);
-}
-
-#ifndef WIN32
-static void
-sighandler(int sig)
-{
- signaled = true;
-}
-
-/* We don't want SIGQUIT to core dump */
-static void
-sigquit_handler(int sig)
-{
- pqsignal(SIGINT, SIG_DFL);
- kill(getpid(), SIGINT);
-}
-#endif
-
-/*------------ MAIN ----------------------------------------*/
-int
-main(int argc, char **argv)
-{
- int c;
-
- progname = get_progname(argv[0]);
-
- if (argc > 1)
- {
- if (strcmp(argv[1], "--help") == 0 || strcmp(argv[1], "-?") == 0)
- {
- usage();
- exit(0);
- }
- if (strcmp(argv[1], "--version") == 0 || strcmp(argv[1], "-V") == 0)
- {
- puts("pg_standby (PostgreSQL) " PG_VERSION);
- exit(0);
- }
- }
-
-#ifndef WIN32
-
- /*
- * You can send SIGUSR1 to trigger failover.
- *
- * Postmaster uses SIGQUIT to request immediate shutdown. The default
- * action is to core dump, but we don't want that, so trap it and commit
- * suicide without core dump.
- *
- * We used to use SIGINT and SIGQUIT to trigger failover, but that turned
- * out to be a bad idea because postmaster uses SIGQUIT to request
- * immediate shutdown. We still trap SIGINT, but that may change in a
- * future release.
- *
- * There's no way to trigger failover via signal on Windows.
- */
- (void) pqsignal(SIGUSR1, sighandler);
- (void) pqsignal(SIGINT, sighandler); /* deprecated, use SIGUSR1 */
- (void) pqsignal(SIGQUIT, sigquit_handler);
-#endif
-
- while ((c = getopt(argc, argv, "cdk:lr:s:t:w:")) != -1)
- {
- switch (c)
- {
- case 'c': /* Use copy */
- restoreCommandType = RESTORE_COMMAND_COPY;
- break;
- case 'd': /* Debug mode */
- debug = true;
- break;
- case 'k': /* keepfiles */
- keepfiles = atoi(optarg);
- if (keepfiles < 0)
- {
- fprintf(stderr, "%s: -k keepfiles must be >= 0\n", progname);
- exit(2);
- }
- break;
- case 'l': /* Use link */
-
- /*
- * Link feature disabled, possibly permanently. Linking causes
- * a problem after recovery ends that is not currently
- * resolved by PostgreSQL. 25 Jun 2009
- */
-#ifdef NOT_USED
- restoreCommandType = RESTORE_COMMAND_LINK;
-#endif
- break;
- case 'r': /* Retries */
- maxretries = atoi(optarg);
- if (maxretries < 0)
- {
- fprintf(stderr, "%s: -r maxretries must be >= 0\n", progname);
- exit(2);
- }
- break;
- case 's': /* Sleep time */
- sleeptime = atoi(optarg);
- if (sleeptime <= 0 || sleeptime > 60)
- {
- fprintf(stderr, "%s: -s sleeptime incorrectly set\n", progname);
- exit(2);
- }
- break;
- case 't': /* Trigger file */
- triggerPath = pg_strdup(optarg);
- break;
- case 'w': /* Max wait time */
- maxwaittime = atoi(optarg);
- if (maxwaittime < 0)
- {
- fprintf(stderr, "%s: -w maxwaittime incorrectly set\n", progname);
- exit(2);
- }
- break;
- default:
- fprintf(stderr, "Try \"%s --help\" for more information.\n", progname);
- exit(2);
- break;
- }
- }
-
- /*
- * Parameter checking - after checking to see if trigger file present
- */
- if (argc == 1)
- {
- fprintf(stderr, "%s: not enough command-line arguments\n", progname);
- exit(2);
- }
-
- /*
- * We will go to the archiveLocation to get nextWALFileName.
- * nextWALFileName may not exist yet, which would not be an error, so we
- * separate the archiveLocation and nextWALFileName so we can check
- * separately whether archiveLocation exists, if not that is an error
- */
- if (optind < argc)
- {
- archiveLocation = argv[optind];
- optind++;
- }
- else
- {
- fprintf(stderr, "%s: must specify archive location\n", progname);
- fprintf(stderr, "Try \"%s --help\" for more information.\n", progname);
- exit(2);
- }
-
- if (optind < argc)
- {
- nextWALFileName = argv[optind];
- optind++;
- }
- else
- {
- fprintf(stderr, "%s: must specify WAL file name as second non-option argument (use \"%%f\")\n", progname);
- fprintf(stderr, "Try \"%s --help\" for more information.\n", progname);
- exit(2);
- }
-
- if (optind < argc)
- {
- xlogFilePath = argv[optind];
- optind++;
- }
- else
- {
- fprintf(stderr, "%s: must specify xlog destination as third non-option argument (use \"%%p\")\n", progname);
- fprintf(stderr, "Try \"%s --help\" for more information.\n", progname);
- exit(2);
- }
-
- if (optind < argc)
- {
- restartWALFileName = argv[optind];
- optind++;
- }
-
- CustomizableInitialize();
-
- if (debug)
- {
- fprintf(stderr, "Trigger file: %s\n", triggerPath ? triggerPath : "<not set>");
- fprintf(stderr, "Waiting for WAL file: %s\n", nextWALFileName);
- fprintf(stderr, "WAL file path: %s\n", WALFilePath);
- fprintf(stderr, "Restoring to: %s\n", xlogFilePath);
- fprintf(stderr, "Sleep interval: %d second%s\n",
- sleeptime, (sleeptime > 1 ? "s" : " "));
- fprintf(stderr, "Max wait interval: %d %s\n",
- maxwaittime, (maxwaittime > 0 ? "seconds" : "forever"));
- fprintf(stderr, "Command for restore: %s\n", restoreCommand);
- fflush(stderr);
- }
-
- /*
- * Check for initial history file: always the first file to be requested
- * It's OK if the file isn't there - all other files need to wait
- */
- if (IsTLHistoryFileName(nextWALFileName))
- {
- nextWALFileType = XLOG_HISTORY;
- if (RestoreWALFileForRecovery())
- exit(0);
- else
- {
- if (debug)
- {
- fprintf(stderr, "history file not found\n");
- fflush(stderr);
- }
- exit(1);
- }
- }
-
- /*
- * Main wait loop
- */
- for (;;)
- {
- /* Check for trigger file or signal first */
- CheckForExternalTrigger();
-#ifndef WIN32
- if (signaled)
- {
- Failover = FastFailover;
- if (debug)
- {
- fprintf(stderr, "signaled to exit: fast failover\n");
- fflush(stderr);
- }
- }
-#endif
-
- /*
- * Check for fast failover immediately, before checking if the
- * requested WAL file is available
- */
- if (Failover == FastFailover)
- exit(1);
-
- if (CustomizableNextWALFileReady())
- {
- /*
- * Once we have restored this file successfully we can remove some
- * prior WAL files. If this restore fails we mustn't remove any
- * file because some of them will be requested again immediately
- * after the failed restore, or when we restart recovery.
- */
- if (RestoreWALFileForRecovery())
- {
- if (need_cleanup)
- CustomizableCleanupPriorWALFiles();
-
- exit(0);
- }
- else
- {
- /* Something went wrong in copying the file */
- exit(1);
- }
- }
-
- /* Check for smart failover if the next WAL file was not available */
- if (Failover == SmartFailover)
- exit(1);
-
- if (sleeptime <= 60)
- pg_usleep(sleeptime * 1000000L);
-
- waittime += sleeptime;
- if (waittime >= maxwaittime && maxwaittime > 0)
- {
- Failover = FastFailover;
- if (debug)
- {
- fprintf(stderr, "Timed out after %d seconds: fast failover\n",
- waittime);
- fflush(stderr);
- }
- }
- if (debug)
- {
- fprintf(stderr, "WAL file not present yet.");
- if (triggerPath)
- fprintf(stderr, " Checking for trigger file...");
- fprintf(stderr, "\n");
- fflush(stderr);
- }
- }
-}
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index ae2759be55..d3ca4b6932 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -192,13 +192,12 @@ pages.
<title>Server Applications</title>
<para>
- This section covers <productname>PostgreSQL</productname> server-related
- applications in <literal>contrib</literal>. They are typically run on the
- host where the database server resides. See also <xref
+ Some applications run on the <productname>PostgreSQL</productname> server
+ itself. Currently, no such applications are included in the
+ <literal>contrib</literal> directory. See also <xref
linkend="reference-server"/> for information about server applications that
are part of the core <productname>PostgreSQL</productname> distribution.
</para>
- &pgstandby;
</sect1>
</appendix>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 38e8aa0bbf..db1d369743 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -138,7 +138,6 @@
<!ENTITY pgfreespacemap SYSTEM "pgfreespacemap.sgml">
<!ENTITY pgprewarm SYSTEM "pgprewarm.sgml">
<!ENTITY pgrowlocks SYSTEM "pgrowlocks.sgml">
-<!ENTITY pgstandby SYSTEM "pgstandby.sgml">
<!ENTITY pgstatstatements SYSTEM "pgstatstatements.sgml">
<!ENTITY pgstattuple SYSTEM "pgstattuple.sgml">
<!ENTITY pgsurgery SYSTEM "pgsurgery.sgml">
diff --git a/doc/src/sgml/high-availability.sgml b/doc/src/sgml/high-availability.sgml
index dc263e4106..f4b02e74f9 100644
--- a/doc/src/sgml/high-availability.sgml
+++ b/doc/src/sgml/high-availability.sgml
@@ -702,8 +702,7 @@ protocol to make nodes agree on a serializable transactional order.
Do not use pg_standby or similar tools with the built-in standby mode
described here. <xref linkend="guc-restore-command"/> should return immediately
if the file does not exist; the server will retry the command again if
- necessary. See <xref linkend="log-shipping-alternative"/>
- for using tools like pg_standby.
+ necessary.
</para>
</note>
@@ -1494,8 +1493,7 @@ synchronous_standby_names = 'ANY 2 (s1, s2, s3)'
<para>
An alternative to the built-in standby mode described in the previous
sections is to use a <varname>restore_command</varname> that polls the archive location.
- This was the only option available in versions 8.4 and below. See the
- <xref linkend="pgstandby"/> module for a reference implementation of this.
+ This was the only option available in versions 8.4 and below.
</para>
<para>
@@ -1551,14 +1549,6 @@ if (!triggered)
</programlisting>
</para>
- <para>
- A working example of a waiting <varname>restore_command</varname> is provided
- in the <xref linkend="pgstandby"/> module. It
- should be used as a reference on how to correctly implement the logic
- described above. It can also be extended as needed to support specific
- configurations and environments.
- </para>
-
<para>
The method for triggering failover is an important part of planning
and design. One potential option is the <varname>restore_command</varname>
diff --git a/doc/src/sgml/pgstandby.sgml b/doc/src/sgml/pgstandby.sgml
deleted file mode 100644
index 66a6255930..0000000000
--- a/doc/src/sgml/pgstandby.sgml
+++ /dev/null
@@ -1,394 +0,0 @@
-<!-- doc/src/sgml/pgstandby.sgml -->
-
-<refentry id="pgstandby">
- <indexterm zone="pgstandby">
- <primary>pg_standby</primary>
- </indexterm>
-
- <refmeta>
- <refentrytitle><application>pg_standby</application></refentrytitle>
- <manvolnum>1</manvolnum>
- <refmiscinfo>Application</refmiscinfo>
- </refmeta>
-
- <refnamediv>
- <refname>pg_standby</refname>
- <refpurpose>supports the creation of a <productname>PostgreSQL</productname> warm standby server</refpurpose>
- </refnamediv>
-
- <refsynopsisdiv>
- <cmdsynopsis>
- <command>pg_standby</command>
- <arg rep="repeat"><replaceable>option</replaceable></arg>
- <arg choice="plain"><replaceable>archivelocation</replaceable></arg>
- <arg choice="plain"><replaceable>nextwalfile</replaceable></arg>
- <arg choice="plain"><replaceable>walfilepath</replaceable></arg>
- <arg choice="opt"><replaceable>restartwalfile</replaceable></arg>
- </cmdsynopsis>
- </refsynopsisdiv>
-
- <refsect1>
- <title>Description</title>
-
- <para>
- <application>pg_standby</application> supports creation of a <quote>warm standby</quote>
- database server. It is designed to be a production-ready program, as well
- as a customizable template should you require specific modifications.
- </para>
-
- <para>
- <application>pg_standby</application> is designed to be a waiting
- <varname>restore_command</varname>, which is needed to turn a standard
- archive recovery into a warm standby operation. Other
- configuration is required as well, all of which is described in the main
- server manual (see <xref linkend="warm-standby"/>).
- </para>
-
- <para>
- To configure a standby
- server to use <application>pg_standby</application>, put this into its
- <filename>postgresql.conf</filename> configuration file:
-<programlisting>
-restore_command = 'pg_standby <replaceable>archiveDir</replaceable> %f %p %r'
-</programlisting>
- where <replaceable>archiveDir</replaceable> is the directory from which WAL segment
- files should be restored.
- </para>
- <para>
- If <replaceable>restartwalfile</replaceable> is specified, normally by using the
- <literal>%r</literal> macro, then all WAL files logically preceding this
- file will be removed from <replaceable>archivelocation</replaceable>. This minimizes
- the number of files that need to be retained, while preserving
- crash-restart capability. Use of this parameter is appropriate if the
- <replaceable>archivelocation</replaceable> is a transient staging area for this
- particular standby server, but <emphasis>not</emphasis> when the
- <replaceable>archivelocation</replaceable> is intended as a long-term WAL archive area.
- </para>
- <para>
- <application>pg_standby</application> assumes that
- <replaceable>archivelocation</replaceable> is a directory readable by the
- server-owning user. If <replaceable>restartwalfile</replaceable> (or <literal>-k</literal>)
- is specified,
- the <replaceable>archivelocation</replaceable> directory must be writable too.
- </para>
- <para>
- There are two ways to fail over to a <quote>warm standby</quote> database server
- when the primary server fails:
-
- <variablelist>
- <varlistentry>
- <term>Smart Failover</term>
- <listitem>
- <para>
- In smart failover, the server is brought up after applying all WAL
- files available in the archive. This results in zero data loss, even if
- the standby server has fallen behind, but if there is a lot of
- unapplied WAL it can be a long time before the standby server becomes
- ready. To trigger a smart failover, create a trigger file containing
- the word <literal>smart</literal>, or just create it and leave it empty.
- </para>
- </listitem>
- </varlistentry>
- <varlistentry>
- <term>Fast Failover</term>
- <listitem>
- <para>
- In fast failover, the server is brought up immediately. Any WAL files
- in the archive that have not yet been applied will be ignored, and
- all transactions in those files are lost. To trigger a fast failover,
- create a trigger file and write the word <literal>fast</literal> into it.
- <application>pg_standby</application> can also be configured to execute a fast
- failover automatically if no new WAL file appears within a defined
- interval.
- </para>
- </listitem>
- </varlistentry>
- </variablelist>
- </para>
-
- </refsect1>
-
- <refsect1>
- <title>Options</title>
-
- <para>
- <application>pg_standby</application> accepts the following command-line arguments:
-
- <variablelist>
-
- <varlistentry>
- <term><option>-c</option></term>
- <listitem>
- <para>
- Use <literal>cp</literal> or <literal>copy</literal> command to restore WAL files
- from archive. This is the only supported behavior so this option is useless.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-d</option></term>
- <listitem>
- <para>
- Print lots of debug logging output on <filename>stderr</filename>.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-k</option></term>
- <listitem>
- <para>
- Remove files from <replaceable>archivelocation</replaceable> so that
- no more than this many WAL files before the current one are kept in the
- archive. Zero (the default) means not to remove any files from
- <replaceable>archivelocation</replaceable>.
- This parameter will be silently ignored if
- <replaceable>restartwalfile</replaceable> is specified, since that
- specification method is more accurate in determining the correct
- archive cut-off point.
- Use of this parameter is <emphasis>deprecated</emphasis> as of
- <productname>PostgreSQL</productname> 8.3; it is safer and more efficient to
- specify a <replaceable>restartwalfile</replaceable> parameter. A too
- small setting could result in removal of files that are still needed
- for a restart of the standby server, while a too large setting wastes
- archive space.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-r</option> <replaceable>maxretries</replaceable></term>
- <listitem>
- <para>
- Set the maximum number of times to retry the copy command if
- it fails (default 3). After each failure, we wait for
- <replaceable>sleeptime</replaceable> * <replaceable>num_retries</replaceable>
- so that the wait time increases progressively. So by default,
- we will wait 5 secs, 10 secs, then 15 secs before reporting
- the failure back to the standby server. This will be
- interpreted as end of recovery and the standby will come
- up fully as a result.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-s</option> <replaceable>sleeptime</replaceable></term>
- <listitem>
- <para>
- Set the number of seconds (up to 60, default 5) to sleep between
- tests to see if the WAL file to be restored is available in
- the archive yet. The default setting is not necessarily
- recommended; consult <xref linkend="warm-standby"/> for discussion.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-t</option> <replaceable>triggerfile</replaceable></term>
- <listitem>
- <para>
- Specify a trigger file whose presence should cause failover.
- It is recommended that you use a structured file name to
- avoid confusion as to which server is being triggered
- when multiple servers exist on the same system; for example
- <filename>/tmp/pgsql.trigger.5432</filename>.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-V</option></term>
- <term><option>--version</option></term>
- <listitem>
- <para>
- Print the <application>pg_standby</application> version and exit.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-w</option> <replaceable>maxwaittime</replaceable></term>
- <listitem>
- <para>
- Set the maximum number of seconds to wait for the next WAL file,
- after which a fast failover will be performed.
- A setting of zero (the default) means wait forever.
- The default setting is not necessarily recommended;
- consult <xref linkend="warm-standby"/> for discussion.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-?</option></term>
- <term><option>--help</option></term>
- <listitem>
- <para>
- Show help about <application>pg_standby</application> command line
- arguments, and exit.
- </para>
- </listitem>
- </varlistentry>
- </variablelist>
- </para>
-
- </refsect1>
-
- <refsect1>
- <title>Notes</title>
-
- <para>
- <application>pg_standby</application> is designed to work with
- <productname>PostgreSQL</productname> 8.2 and later.
- </para>
- <para>
- <productname>PostgreSQL</productname> 8.3 provides the <literal>%r</literal> macro,
- which is designed to let <application>pg_standby</application> know the
- last file it needs to keep. With <productname>PostgreSQL</productname> 8.2, the
- <literal>-k</literal> option must be used if archive cleanup is
- required. This option remains available in 8.3, but its use is deprecated.
- </para>
- <para>
- <productname>PostgreSQL</productname> 8.4 provides the
- <varname>recovery_end_command</varname> option. Without this option
- a leftover trigger file can be hazardous.
- </para>
-
- <para>
- <application>pg_standby</application> is written in C and has an
- easy-to-modify source code, with specifically designated sections to modify
- for your own needs
- </para>
- </refsect1>
-
- <refsect1>
- <title>Examples</title>
-
- <para>On Linux or Unix systems, you might use:
-
-<programlisting>
-archive_command = 'cp %p .../archive/%f'
-
-restore_command = 'pg_standby -d -s 2 -t /tmp/pgsql.trigger.5442 .../archive %f %p %r 2>>standby.log'
-
-recovery_end_command = 'rm -f /tmp/pgsql.trigger.5442'
-</programlisting>
- where the archive directory is physically located on the standby server,
- so that the <varname>archive_command</varname> is accessing it across NFS,
- but the files are local to the standby (enabling use of <literal>ln</literal>).
- This will:
- <itemizedlist>
- <listitem>
- <para>
- produce debugging output in <filename>standby.log</filename>
- </para>
- </listitem>
- <listitem>
- <para>
- sleep for 2 seconds between checks for next WAL file availability
- </para>
- </listitem>
- <listitem>
- <para>
- stop waiting only when a trigger file called
- <filename>/tmp/pgsql.trigger.5442</filename> appears,
- and perform failover according to its content
- </para>
- </listitem>
- <listitem>
- <para>
- remove the trigger file when recovery ends
- </para>
- </listitem>
- <listitem>
- <para>
- remove no-longer-needed files from the archive directory
- </para>
- </listitem>
- </itemizedlist>
- </para>
-
- <para>On Windows, you might use:
-
-<programlisting>
-archive_command = 'copy %p ...\\archive\\%f'
-
-restore_command = 'pg_standby -d -s 5 -t C:\pgsql.trigger.5442 ...\archive %f %p %r 2>>standby.log'
-
-recovery_end_command = 'del C:\pgsql.trigger.5442'
-</programlisting>
- Note that backslashes need to be doubled in the
- <varname>archive_command</varname>, but <emphasis>not</emphasis> in the
- <varname>restore_command</varname> or <varname>recovery_end_command</varname>.
- This will:
- <itemizedlist>
- <listitem>
- <para>
- use the <literal>copy</literal> command to restore WAL files from archive
- </para>
- </listitem>
- <listitem>
- <para>
- produce debugging output in <filename>standby.log</filename>
- </para>
- </listitem>
- <listitem>
- <para>
- sleep for 5 seconds between checks for next WAL file availability
- </para>
- </listitem>
- <listitem>
- <para>
- stop waiting only when a trigger file called
- <filename>C:\pgsql.trigger.5442</filename> appears,
- and perform failover according to its content
- </para>
- </listitem>
- <listitem>
- <para>
- remove the trigger file when recovery ends
- </para>
- </listitem>
- <listitem>
- <para>
- remove no-longer-needed files from the archive directory
- </para>
- </listitem>
- </itemizedlist>
- </para>
-
- <para>
- The <literal>copy</literal> command on Windows sets the final file size
- before the file is completely copied, which would ordinarily confuse
- <application>pg_standby</application>. Therefore
- <application>pg_standby</application> waits <replaceable>sleeptime</replaceable>
- seconds once it sees the proper file size. GNUWin32's <literal>cp</literal>
- sets the file size only after the file copy is complete.
- </para>
-
- <para>
- Since the Windows example uses <literal>copy</literal> at both ends, either
- or both servers might be accessing the archive directory across the
- network.
- </para>
-
- </refsect1>
-
- <refsect1>
- <title>Author</title>
-
- <para>
- Simon Riggs <email>simon@2ndquadrant.com</email>
- </para>
- </refsect1>
-
- <refsect1>
- <title>See Also</title>
-
- <simplelist type="inline">
- <member><xref linkend="pgarchivecleanup"/></member>
- </simplelist>
- </refsect1>
-</refentry>
diff --git a/doc/src/sgml/ref/pgarchivecleanup.sgml b/doc/src/sgml/ref/pgarchivecleanup.sgml
index 56f02fc0e6..e27db3c077 100644
--- a/doc/src/sgml/ref/pgarchivecleanup.sgml
+++ b/doc/src/sgml/ref/pgarchivecleanup.sgml
@@ -205,11 +205,4 @@ archive_cleanup_command = 'pg_archivecleanup -d /mnt/standby/archive %r 2>>clean
</itemizedlist>
</refsect1>
- <refsect1>
- <title>See Also</title>
-
- <simplelist type="inline">
- <member><xref linkend="pgstandby"/></member>
- </simplelist>
- </refsect1>
</refentry>
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 7213e65e08..90328db04e 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -34,8 +34,8 @@ my @unlink_on_exit;
# Set of variables for modules in contrib/ and src/test/modules/
my $contrib_defines = { 'refint' => 'REFINT_VERBOSE' };
my @contrib_uselibpq = ('dblink', 'oid2name', 'postgres_fdw', 'vacuumlo');
-my @contrib_uselibpgport = ('oid2name', 'pg_standby', 'vacuumlo');
-my @contrib_uselibpgcommon = ('oid2name', 'pg_standby', 'vacuumlo');
+my @contrib_uselibpgport = ('oid2name', 'vacuumlo');
+my @contrib_uselibpgcommon = ('oid2name', 'vacuumlo');
my $contrib_extralibs = undef;
my $contrib_extraincludes = { 'dblink' => ['src/backend'] };
my $contrib_extrasource = {
--
2.20.1
v3-0002-Remove-documentation-of-waiting-restore_command.patch (text/x-patch, 6.8K) download
From f6541076dde7d8fb347ee618b33734d241bfa609 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 27 Jan 2021 16:04:55 +1300
Subject: [PATCH v3 2/2] Remove documentation of waiting restore_command.
Following the removal of pg_standby, also remove the documentation
section that describes how to write your own "waiting restore command".
Such restore commands do not interact well with a propose recovery
prefetching feature.
Discussion: https://postgr.es/m/20201029024412.GP5380%40telsasoft.com
---
doc/src/sgml/high-availability.sgml | 137 ----------------------------
1 file changed, 137 deletions(-)
diff --git a/doc/src/sgml/high-availability.sgml b/doc/src/sgml/high-availability.sgml
index f4b02e74f9..f9736bcbc1 100644
--- a/doc/src/sgml/high-availability.sgml
+++ b/doc/src/sgml/high-availability.sgml
@@ -1487,143 +1487,6 @@ synchronous_standby_names = 'ANY 2 (s1, s2, s3)'
</para>
</sect1>
- <sect1 id="log-shipping-alternative">
- <title>Alternative Method for Log Shipping</title>
-
- <para>
- An alternative to the built-in standby mode described in the previous
- sections is to use a <varname>restore_command</varname> that polls the archive location.
- This was the only option available in versions 8.4 and below.
- </para>
-
- <para>
- Note that in this mode, the server will apply WAL one file at a
- time, so if you use the standby server for queries (see Hot Standby),
- there is a delay between an action in the primary and when the
- action becomes visible in the standby, corresponding to the time it takes
- to fill up the WAL file. <varname>archive_timeout</varname> can be used to make that delay
- shorter. Also note that you can't combine streaming replication with
- this method.
- </para>
-
- <para>
- The operations that occur on both primary and standby servers are
- normal continuous archiving and recovery tasks. The only point of
- contact between the two database servers is the archive of WAL files
- that both share: primary writing to the archive, standby reading from
- the archive. Care must be taken to ensure that WAL archives from separate
- primary servers do not become mixed together or confused. The archive
- need not be large if it is only required for standby operation.
- </para>
-
- <para>
- The magic that makes the two loosely coupled servers work together is
- simply a <varname>restore_command</varname> used on the standby that,
- when asked for the next WAL file, waits for it to become available from
- the primary. Normal recovery
- processing would request a file from the WAL archive, reporting failure
- if the file was unavailable. For standby processing it is normal for
- the next WAL file to be unavailable, so the standby must wait for
- it to appear. For files ending in
- <literal>.history</literal> there is no need to wait, and a non-zero return
- code must be returned. A waiting <varname>restore_command</varname> can be
- written as a custom script that loops after polling for the existence of
- the next WAL file. There must also be some way to trigger failover, which
- should interrupt the <varname>restore_command</varname>, break the loop and
- return a file-not-found error to the standby server. This ends recovery
- and the standby will then come up as a normal server.
- </para>
-
- <para>
- Pseudocode for a suitable <varname>restore_command</varname> is:
-<programlisting>
-triggered = false;
-while (!NextWALFileReady() && !triggered)
-{
- sleep(100000L); /* wait for ~0.1 sec */
- if (CheckForExternalTrigger())
- triggered = true;
-}
-if (!triggered)
- CopyWALFileForRecovery();
-</programlisting>
- </para>
-
- <para>
- The method for triggering failover is an important part of planning
- and design. One potential option is the <varname>restore_command</varname>
- command. It is executed once for each WAL file, but the process
- running the <varname>restore_command</varname> is created and dies for
- each file, so there is no daemon or server process, and
- signals or a signal handler cannot be used. Therefore, the
- <varname>restore_command</varname> is not suitable to trigger failover.
- It is possible to use a simple timeout facility, especially if
- used in conjunction with a known <varname>archive_timeout</varname>
- setting on the primary. However, this is somewhat error prone
- since a network problem or busy primary server might be sufficient
- to initiate failover. A notification mechanism such as the explicit
- creation of a trigger file is ideal, if this can be arranged.
- </para>
-
- <sect2 id="warm-standby-config">
- <title>Implementation</title>
-
- <para>
- The short procedure for configuring a standby server using this alternative
- method is as follows. For
- full details of each step, refer to previous sections as noted.
- <orderedlist>
- <listitem>
- <para>
- Set up primary and standby systems as nearly identical as
- possible, including two identical copies of
- <productname>PostgreSQL</productname> at the same release level.
- </para>
- </listitem>
- <listitem>
- <para>
- Set up continuous archiving from the primary to a WAL archive
- directory on the standby server. Ensure that
- <xref linkend="guc-archive-mode"/>,
- <xref linkend="guc-archive-command"/> and
- <xref linkend="guc-archive-timeout"/>
- are set appropriately on the primary
- (see <xref linkend="backup-archiving-wal"/>).
- </para>
- </listitem>
- <listitem>
- <para>
- Make a base backup of the primary server (see <xref
- linkend="backup-base-backup"/>), and load this data onto the standby.
- </para>
- </listitem>
- <listitem>
- <para>
- Begin recovery on the standby server from the local WAL
- archive, using <varname>restore_command</varname> that waits
- as described previously (see <xref linkend="backup-pitr-recovery"/>).
- </para>
- </listitem>
- </orderedlist>
- </para>
-
- <para>
- Recovery treats the WAL archive as read-only, so once a WAL file has
- been copied to the standby system it can be copied to tape at the same
- time as it is being read by the standby database server.
- Thus, running a standby server for high availability can be performed at
- the same time as files are stored for longer term disaster recovery
- purposes.
- </para>
-
- <para>
- For testing purposes, it is possible to run both primary and standby
- servers on the same system. This does not provide any worthwhile
- improvement in server robustness, nor would it be described as HA.
- </para>
- </sect2>
- </sect1>
-
<sect1 id="hot-standby">
<title>Hot Standby</title>
--
2.20.1
* Re: [PATCH] remove pg_standby
2020-11-21 19:41 Re: [PATCH] remove pg_standby Justin Pryzby <pryzby@telsasoft.com>
2020-11-25 09:04 ` Re: [PATCH] remove pg_standby Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2021-01-27 03:13 ` Re: [PATCH] remove pg_standby Thomas Munro <thomas.munro@gmail.com>
@ 2021-01-27 05:06 ` Michael Paquier <michael@paquier.xyz>
2021-01-27 05:32 ` Re: [PATCH] remove pg_standby Thomas Munro <thomas.munro@gmail.com>
0 siblings, 1 reply; 25+ messages in thread
From: Michael Paquier @ 2021-01-27 05:06 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: Peter Eisentraut <peter.eisentraut@enterprisedb.com>; Justin Pryzby <pryzby@telsasoft.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Tom Lane <tgl@sss.pgh.pa.us>; Stephen Frost <sfrost@snowman.net>; Ian Lawrence Barwick <barwick@gmail.com>; pgsql-hackers
On Wed, Jan 27, 2021 at 04:13:24PM +1300, Thomas Munro wrote:
> I would like to commit this, because "waiting restore commands" have
> confusing interactions with my proposed prefetching-during-recovery
> patch[1]. Here's a version that fixes an error when building the docs
> (there was a stray remaining <xref linkend="pgstandby"/>), and adds a
> commit message. Any objections?
It looks like you are missing two references in your patch set:
$ git grep pg_standby
doc/src/sgml/high-availability.sgml: Do not use pg_standby or
similar tools with the built-in standby mode
src/backend/access/transam/xlog.c: * segment. Only recycle normal
files, pg_standby for example can create
The logic assumed in RemoveXlogFile() is actually a bit scary. I have
not checked in details but it could be possible to clean up more code
in this area?
> Furthermore, I think we should also remove the section of the manual
> that describes how to write your own "waiting restore command".
> Thoughts?
Agreed. No objections to that.
--
Michael
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: [PATCH] remove pg_standby
2020-11-21 19:41 Re: [PATCH] remove pg_standby Justin Pryzby <pryzby@telsasoft.com>
2020-11-25 09:04 ` Re: [PATCH] remove pg_standby Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2021-01-27 03:13 ` Re: [PATCH] remove pg_standby Thomas Munro <thomas.munro@gmail.com>
2021-01-27 05:06 ` Re: [PATCH] remove pg_standby Michael Paquier <michael@paquier.xyz>
@ 2021-01-27 05:32 ` Thomas Munro <thomas.munro@gmail.com>
2021-01-27 08:08 ` Re: [PATCH] remove pg_standby Fujii Masao <masao.fujii@oss.nttdata.com>
0 siblings, 1 reply; 25+ messages in thread
From: Thomas Munro @ 2021-01-27 05:32 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: Peter Eisentraut <peter.eisentraut@enterprisedb.com>; Justin Pryzby <pryzby@telsasoft.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Tom Lane <tgl@sss.pgh.pa.us>; Stephen Frost <sfrost@snowman.net>; Ian Lawrence Barwick <barwick@gmail.com>; pgsql-hackers
On Wed, Jan 27, 2021 at 6:06 PM Michael Paquier <michael@paquier.xyz> wrote:
> On Wed, Jan 27, 2021 at 04:13:24PM +1300, Thomas Munro wrote:
> > I would like to commit this, because "waiting restore commands" have
> > confusing interactions with my proposed prefetching-during-recovery
> > patch[1]. Here's a version that fixes an error when building the docs
> > (there was a stray remaining <xref linkend="pgstandby"/>), and adds a
> > commit message. Any objections?
>
> It looks like you are missing two references in your patch set:
> $ git grep pg_standby
> doc/src/sgml/high-availability.sgml: Do not use pg_standby or
> similar tools with the built-in standby mode
> src/backend/access/transam/xlog.c: * segment. Only recycle normal
> files, pg_standby for example can create
Thanks, fixed.
> The logic assumed in RemoveXlogFile() is actually a bit scary. I have
> not checked in details but it could be possible to clean up more code
> in this area?
I think the check that it's a regular file is a good idea anyway, but
I removed the offending comment.
> > Furthermore, I think we should also remove the section of the manual
> > that describes how to write your own "waiting restore command".
> > Thoughts?
>
> Agreed. No objections to that.
Thanks!
Attachments: download all (2 files, .tar.gz)
v4-0001-Retire-pg_standby.patch (text/x-patch, 45.5K) download
From 77015331a71e67a4dee5352afc9f5faa05524755 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Mon, 26 Oct 2020 16:37:46 -0500
Subject: [PATCH v4 1/2] Retire pg_standby.
Streaming replication made pg_standby obsolete. It has been proposed
that we retire it many times. Now seems like a good time to do it,
because "waiting restore commands" are incompatible with a proposed
recovery prefetching feature.
Discussion: https://postgr.es/m/20201029024412.GP5380%40telsasoft.com
Author: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
---
contrib/Makefile | 1 -
contrib/pg_standby/.gitignore | 1 -
contrib/pg_standby/Makefile | 20 -
contrib/pg_standby/pg_standby.c | 907 -------------------------
doc/src/sgml/contrib.sgml | 7 +-
doc/src/sgml/filelist.sgml | 1 -
doc/src/sgml/high-availability.sgml | 17 +-
doc/src/sgml/pgstandby.sgml | 394 -----------
doc/src/sgml/ref/pgarchivecleanup.sgml | 7 -
src/backend/access/transam/xlog.c | 2 +-
src/tools/msvc/Mkvcbuild.pm | 4 +-
11 files changed, 9 insertions(+), 1352 deletions(-)
delete mode 100644 contrib/pg_standby/.gitignore
delete mode 100644 contrib/pg_standby/Makefile
delete mode 100644 contrib/pg_standby/pg_standby.c
delete mode 100644 doc/src/sgml/pgstandby.sgml
diff --git a/contrib/Makefile b/contrib/Makefile
index 7a4866e338..cdc041c7db 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -33,7 +33,6 @@ SUBDIRS = \
pg_buffercache \
pg_freespacemap \
pg_prewarm \
- pg_standby \
pg_stat_statements \
pg_surgery \
pg_trgm \
diff --git a/contrib/pg_standby/.gitignore b/contrib/pg_standby/.gitignore
deleted file mode 100644
index a401b085a8..0000000000
--- a/contrib/pg_standby/.gitignore
+++ /dev/null
@@ -1 +0,0 @@
-/pg_standby
diff --git a/contrib/pg_standby/Makefile b/contrib/pg_standby/Makefile
deleted file mode 100644
index 87732bedf1..0000000000
--- a/contrib/pg_standby/Makefile
+++ /dev/null
@@ -1,20 +0,0 @@
-# contrib/pg_standby/Makefile
-
-PGFILEDESC = "pg_standby - supports creation of a warm standby"
-PGAPPICON = win32
-
-PROGRAM = pg_standby
-OBJS = \
- $(WIN32RES) \
- pg_standby.o
-
-ifdef USE_PGXS
-PG_CONFIG = pg_config
-PGXS := $(shell $(PG_CONFIG) --pgxs)
-include $(PGXS)
-else
-subdir = contrib/pg_standby
-top_builddir = ../..
-include $(top_builddir)/src/Makefile.global
-include $(top_srcdir)/contrib/contrib-global.mk
-endif
diff --git a/contrib/pg_standby/pg_standby.c b/contrib/pg_standby/pg_standby.c
deleted file mode 100644
index c9f33e4254..0000000000
--- a/contrib/pg_standby/pg_standby.c
+++ /dev/null
@@ -1,907 +0,0 @@
-/*
- * contrib/pg_standby/pg_standby.c
- *
- *
- * pg_standby.c
- *
- * Production-ready example of how to create a Warm Standby
- * database server using continuous archiving as a
- * replication mechanism
- *
- * We separate the parameters for archive and nextWALfile
- * so that we can check the archive exists, even if the
- * WAL file doesn't (yet).
- *
- * This program will be executed once in full for each file
- * requested by the warm standby server.
- *
- * It is designed to cater to a variety of needs, as well
- * providing a customizable section.
- *
- * Original author: Simon Riggs simon@2ndquadrant.com
- * Current maintainer: Simon Riggs
- */
-#include "postgres_fe.h"
-
-#include <ctype.h>
-#include <dirent.h>
-#include <sys/stat.h>
-#include <fcntl.h>
-#include <signal.h>
-#include <sys/time.h>
-
-#include "access/xlog_internal.h"
-#include "pg_getopt.h"
-
-const char *progname;
-
-int WalSegSz = -1;
-
-/* Options and defaults */
-int sleeptime = 5; /* amount of time to sleep between file checks */
-int waittime = -1; /* how long we have been waiting, -1 no wait
- * yet */
-int maxwaittime = 0; /* how long are we prepared to wait for? */
-int keepfiles = 0; /* number of WAL files to keep, 0 keep all */
-int maxretries = 3; /* number of retries on restore command */
-bool debug = false; /* are we debugging? */
-bool need_cleanup = false; /* do we need to remove files from
- * archive? */
-
-#ifndef WIN32
-static volatile sig_atomic_t signaled = false;
-#endif
-
-char *archiveLocation; /* where to find the archive? */
-char *triggerPath; /* where to find the trigger file? */
-char *xlogFilePath; /* where we are going to restore to */
-char *nextWALFileName; /* the file we need to get from archive */
-char *restartWALFileName; /* the file from which we can restart restore */
-char WALFilePath[MAXPGPATH * 2]; /* the file path including archive */
-char restoreCommand[MAXPGPATH]; /* run this to restore */
-char exclusiveCleanupFileName[MAXFNAMELEN]; /* the file we need to get
- * from archive */
-
-/*
- * Two types of failover are supported (smart and fast failover).
- *
- * The content of the trigger file determines the type of failover. If the
- * trigger file contains the word "smart" (or the file is empty), smart
- * failover is chosen: pg_standby acts as cp or ln command itself, on
- * successful completion all the available WAL records will be applied
- * resulting in zero data loss. But, it might take a long time to finish
- * recovery if there's a lot of unapplied WAL.
- *
- * On the other hand, if the trigger file contains the word "fast", the
- * recovery is finished immediately even if unapplied WAL files remain. Any
- * transactions in the unapplied WAL files are lost.
- *
- * An empty trigger file performs smart failover. SIGUSR or SIGINT triggers
- * fast failover. A timeout causes fast failover (smart failover would have
- * the same effect, since if the timeout is reached there is no unapplied WAL).
- */
-#define NoFailover 0
-#define SmartFailover 1
-#define FastFailover 2
-
-static int Failover = NoFailover;
-
-#define RESTORE_COMMAND_COPY 0
-#define RESTORE_COMMAND_LINK 1
-int restoreCommandType;
-
-#define XLOG_DATA 0
-#define XLOG_HISTORY 1
-int nextWALFileType;
-
-#define SET_RESTORE_COMMAND(cmd, arg1, arg2) \
- snprintf(restoreCommand, MAXPGPATH, cmd " \"%s\" \"%s\"", arg1, arg2)
-
-struct stat stat_buf;
-
-static bool SetWALFileNameForCleanup(void);
-static bool SetWALSegSize(void);
-
-
-/* =====================================================================
- *
- * Customizable section
- *
- * =====================================================================
- *
- * Currently, this section assumes that the Archive is a locally
- * accessible directory. If you want to make other assumptions,
- * such as using a vendor-specific archive and access API, these
- * routines are the ones you'll need to change. You're
- * encouraged to submit any changes to pgsql-hackers@lists.postgresql.org
- * or personally to the current maintainer. Those changes may be
- * folded in to later versions of this program.
- */
-
-/*
- * Initialize allows customized commands into the warm standby program.
- *
- * As an example, and probably the common case, we use either
- * cp/ln commands on *nix, or copy/move command on Windows.
- */
-static void
-CustomizableInitialize(void)
-{
-#ifdef WIN32
- snprintf(WALFilePath, MAXPGPATH, "%s\\%s", archiveLocation, nextWALFileName);
- switch (restoreCommandType)
- {
- case RESTORE_COMMAND_LINK:
- SET_RESTORE_COMMAND("mklink", WALFilePath, xlogFilePath);
- break;
- case RESTORE_COMMAND_COPY:
- default:
- SET_RESTORE_COMMAND("copy", WALFilePath, xlogFilePath);
- break;
- }
-#else
- snprintf(WALFilePath, MAXPGPATH, "%s/%s", archiveLocation, nextWALFileName);
- switch (restoreCommandType)
- {
- case RESTORE_COMMAND_LINK:
- SET_RESTORE_COMMAND("ln -s -f", WALFilePath, xlogFilePath);
- break;
- case RESTORE_COMMAND_COPY:
- default:
- SET_RESTORE_COMMAND("cp", WALFilePath, xlogFilePath);
- break;
- }
-#endif
-
- /*
- * This code assumes that archiveLocation is a directory You may wish to
- * add code to check for tape libraries, etc.. So, since it is a
- * directory, we use stat to test if it's accessible
- */
- if (stat(archiveLocation, &stat_buf) != 0)
- {
- fprintf(stderr, "%s: archive location \"%s\" does not exist\n", progname, archiveLocation);
- fflush(stderr);
- exit(2);
- }
-}
-
-/*
- * CustomizableNextWALFileReady()
- *
- * Is the requested file ready yet?
- */
-static bool
-CustomizableNextWALFileReady(void)
-{
- if (stat(WALFilePath, &stat_buf) == 0)
- {
- /*
- * If we've not seen any WAL segments, we don't know the WAL segment
- * size, which we need. If it looks like a WAL segment, determine size
- * of segments for the cluster.
- */
- if (WalSegSz == -1 && IsXLogFileName(nextWALFileName))
- {
- if (SetWALSegSize())
- {
- /*
- * Successfully determined WAL segment size. Can compute
- * cleanup cutoff now.
- */
- need_cleanup = SetWALFileNameForCleanup();
- if (debug)
- {
- fprintf(stderr,
- _("WAL segment size: %d \n"), WalSegSz);
- fprintf(stderr, "Keep archive history: ");
-
- if (need_cleanup)
- fprintf(stderr, "%s and later\n",
- exclusiveCleanupFileName);
- else
- fprintf(stderr, "no cleanup required\n");
- }
- }
- }
-
- /*
- * Return only if it's the right size already.
- */
- if (WalSegSz > 0 && stat_buf.st_size == WalSegSz)
- {
-#ifdef WIN32
-
- /*
- * Windows 'cp' sets the final file size before the copy is
- * complete, and not yet ready to be opened by pg_standby. So we
- * wait for sleeptime secs before attempting to restore. If that
- * is not enough, we will rely on the retry/holdoff mechanism.
- * GNUWin32's cp does not have this problem.
- */
- pg_usleep(sleeptime * 1000000L);
-#endif
- nextWALFileType = XLOG_DATA;
- return true;
- }
-
- /*
- * If still too small, wait until it is the correct size
- */
- if (WalSegSz > 0 && stat_buf.st_size > WalSegSz)
- {
- if (debug)
- {
- fprintf(stderr, "file size greater than expected\n");
- fflush(stderr);
- }
- exit(3);
- }
- }
-
- return false;
-}
-
-static void
-CustomizableCleanupPriorWALFiles(void)
-{
- /*
- * Work out name of prior file from current filename
- */
- if (nextWALFileType == XLOG_DATA)
- {
- int rc;
- DIR *xldir;
- struct dirent *xlde;
-
- /*
- * Assume it's OK to keep failing. The failure situation may change
- * over time, so we'd rather keep going on the main processing than
- * fail because we couldn't clean up yet.
- */
- if ((xldir = opendir(archiveLocation)) != NULL)
- {
- while (errno = 0, (xlde = readdir(xldir)) != NULL)
- {
- /*
- * We ignore the timeline part of the XLOG segment identifiers
- * in deciding whether a segment is still needed. This
- * ensures that we won't prematurely remove a segment from a
- * parent timeline. We could probably be a little more
- * proactive about removing segments of non-parent timelines,
- * but that would be a whole lot more complicated.
- *
- * We use the alphanumeric sorting property of the filenames
- * to decide which ones are earlier than the
- * exclusiveCleanupFileName file. Note that this means files
- * are not removed in the order they were originally written,
- * in case this worries you.
- */
- if (IsXLogFileName(xlde->d_name) &&
- strcmp(xlde->d_name + 8, exclusiveCleanupFileName + 8) < 0)
- {
-#ifdef WIN32
- snprintf(WALFilePath, sizeof(WALFilePath), "%s\\%s", archiveLocation, xlde->d_name);
-#else
- snprintf(WALFilePath, sizeof(WALFilePath), "%s/%s", archiveLocation, xlde->d_name);
-#endif
-
- if (debug)
- fprintf(stderr, "\nremoving file \"%s\"", WALFilePath);
-
- rc = unlink(WALFilePath);
- if (rc != 0)
- {
- fprintf(stderr, "\n%s: ERROR: could not remove file \"%s\": %s\n",
- progname, WALFilePath, strerror(errno));
- break;
- }
- }
- }
-
- if (errno)
- fprintf(stderr, "%s: could not read archive location \"%s\": %s\n",
- progname, archiveLocation, strerror(errno));
- if (debug)
- fprintf(stderr, "\n");
- }
- else
- fprintf(stderr, "%s: could not open archive location \"%s\": %s\n",
- progname, archiveLocation, strerror(errno));
-
- if (closedir(xldir))
- fprintf(stderr, "%s: could not close archive location \"%s\": %s\n",
- progname, archiveLocation, strerror(errno));
-
- fflush(stderr);
- }
-}
-
-/* =====================================================================
- * End of Customizable section
- * =====================================================================
- */
-
-/*
- * SetWALFileNameForCleanup()
- *
- * Set the earliest WAL filename that we want to keep on the archive
- * and decide whether we need_cleanup
- */
-static bool
-SetWALFileNameForCleanup(void)
-{
- uint32 tli = 1,
- log = 0,
- seg = 0;
- uint32 log_diff = 0,
- seg_diff = 0;
- bool cleanup = false;
- int max_segments_per_logfile = (0xFFFFFFFF / WalSegSz);
-
- if (restartWALFileName)
- {
- /*
- * Don't do cleanup if the restartWALFileName provided is later than
- * the xlog file requested. This is an error and we must not remove
- * these files from archive. This shouldn't happen, but better safe
- * than sorry.
- */
- if (strcmp(restartWALFileName, nextWALFileName) > 0)
- return false;
-
- strlcpy(exclusiveCleanupFileName, restartWALFileName, sizeof(exclusiveCleanupFileName));
- return true;
- }
-
- if (keepfiles > 0)
- {
- sscanf(nextWALFileName, "%08X%08X%08X", &tli, &log, &seg);
- if (tli > 0 && seg > 0)
- {
- log_diff = keepfiles / max_segments_per_logfile;
- seg_diff = keepfiles % max_segments_per_logfile;
- if (seg_diff > seg)
- {
- log_diff++;
- seg = max_segments_per_logfile - (seg_diff - seg);
- }
- else
- seg -= seg_diff;
-
- if (log >= log_diff)
- {
- log -= log_diff;
- cleanup = true;
- }
- else
- {
- log = 0;
- seg = 0;
- }
- }
- }
-
- XLogFileNameById(exclusiveCleanupFileName, tli, log, seg);
-
- return cleanup;
-}
-
-/*
- * Try to set the wal segment size from the WAL file specified by WALFilePath.
- *
- * Return true if size could be determined, false otherwise.
- */
-static bool
-SetWALSegSize(void)
-{
- bool ret_val = false;
- int fd;
- PGAlignedXLogBlock buf;
-
- Assert(WalSegSz == -1);
-
- if ((fd = open(WALFilePath, O_RDWR, 0)) < 0)
- {
- fprintf(stderr, "%s: could not open WAL file \"%s\": %s\n",
- progname, WALFilePath, strerror(errno));
- return false;
- }
-
- errno = 0;
- if (read(fd, buf.data, XLOG_BLCKSZ) == XLOG_BLCKSZ)
- {
- XLogLongPageHeader longhdr = (XLogLongPageHeader) buf.data;
-
- WalSegSz = longhdr->xlp_seg_size;
-
- if (IsValidWalSegSize(WalSegSz))
- {
- /* successfully retrieved WAL segment size */
- ret_val = true;
- }
- else
- fprintf(stderr,
- "%s: WAL segment size must be a power of two between 1MB and 1GB, but the WAL file header specifies %d bytes\n",
- progname, WalSegSz);
- }
- else
- {
- /*
- * Don't complain loudly, this is to be expected for segments being
- * created.
- */
- if (errno != 0)
- {
- if (debug)
- fprintf(stderr, "could not read file \"%s\": %s\n",
- WALFilePath, strerror(errno));
- }
- else
- {
- if (debug)
- fprintf(stderr, "not enough data in file \"%s\"\n",
- WALFilePath);
- }
- }
-
- fflush(stderr);
-
- close(fd);
- return ret_val;
-}
-
-/*
- * CheckForExternalTrigger()
- *
- * Is there a trigger file? Sets global 'Failover' variable to indicate
- * what kind of a trigger file it was. A "fast" trigger file is turned
- * into a "smart" file as a side-effect.
- */
-static void
-CheckForExternalTrigger(void)
-{
- char buf[32];
- int fd;
- int len;
-
- /*
- * Look for a trigger file, if that option has been selected
- *
- * We use stat() here because triggerPath is always a file rather than
- * potentially being in an archive
- */
- if (!triggerPath || stat(triggerPath, &stat_buf) != 0)
- return;
-
- /*
- * An empty trigger file performs smart failover. There's a little race
- * condition here: if the writer of the trigger file has just created the
- * file, but not yet written anything to it, we'll treat that as smart
- * shutdown even if the other process was just about to write "fast" to
- * it. But that's fine: we'll restore one more WAL file, and when we're
- * invoked next time, we'll see the word "fast" and fail over immediately.
- */
- if (stat_buf.st_size == 0)
- {
- Failover = SmartFailover;
- fprintf(stderr, "trigger file found: smart failover\n");
- fflush(stderr);
- return;
- }
-
- if ((fd = open(triggerPath, O_RDWR, 0)) < 0)
- {
- fprintf(stderr, "WARNING: could not open \"%s\": %s\n",
- triggerPath, strerror(errno));
- fflush(stderr);
- return;
- }
-
- if ((len = read(fd, buf, sizeof(buf) - 1)) < 0)
- {
- fprintf(stderr, "WARNING: could not read \"%s\": %s\n",
- triggerPath, strerror(errno));
- fflush(stderr);
- close(fd);
- return;
- }
- buf[len] = '\0';
-
- if (strncmp(buf, "smart", 5) == 0)
- {
- Failover = SmartFailover;
- fprintf(stderr, "trigger file found: smart failover\n");
- fflush(stderr);
- close(fd);
- return;
- }
-
- if (strncmp(buf, "fast", 4) == 0)
- {
- Failover = FastFailover;
-
- fprintf(stderr, "trigger file found: fast failover\n");
- fflush(stderr);
-
- /*
- * Turn it into a "smart" trigger by truncating the file. Otherwise if
- * the server asks us again to restore a segment that was restored
- * already, we would return "not found" and upset the server.
- */
- if (ftruncate(fd, 0) < 0)
- {
- fprintf(stderr, "WARNING: could not read \"%s\": %s\n",
- triggerPath, strerror(errno));
- fflush(stderr);
- }
- close(fd);
-
- return;
- }
- close(fd);
-
- fprintf(stderr, "WARNING: invalid content in \"%s\"\n", triggerPath);
- fflush(stderr);
-}
-
-/*
- * RestoreWALFileForRecovery()
- *
- * Perform the action required to restore the file from archive
- */
-static bool
-RestoreWALFileForRecovery(void)
-{
- int rc = 0;
- int numretries = 0;
-
- if (debug)
- {
- fprintf(stderr, "running restore: ");
- fflush(stderr);
- }
-
- while (numretries <= maxretries)
- {
- rc = system(restoreCommand);
- if (rc == 0)
- {
- if (debug)
- {
- fprintf(stderr, "OK\n");
- fflush(stderr);
- }
- return true;
- }
- pg_usleep(numretries++ * sleeptime * 1000000L);
- }
-
- /*
- * Allow caller to add additional info
- */
- if (debug)
- fprintf(stderr, "not restored\n");
- return false;
-}
-
-static void
-usage(void)
-{
- printf("%s allows PostgreSQL warm standby servers to be configured.\n\n", progname);
- printf("Usage:\n");
- printf(" %s [OPTION]... ARCHIVELOCATION NEXTWALFILE XLOGFILEPATH [RESTARTWALFILE]\n", progname);
- printf("\nOptions:\n");
- printf(" -c copy file from archive (default)\n");
- printf(" -d generate lots of debugging output (testing only)\n");
- printf(" -k NUMFILESTOKEEP if RESTARTWALFILE is not used, remove files prior to limit\n"
- " (0 keeps all)\n");
- printf(" -l does nothing; use of link is now deprecated\n");
- printf(" -r MAXRETRIES max number of times to retry, with progressive wait\n"
- " (default=3)\n");
- printf(" -s SLEEPTIME seconds to wait between file checks (min=1, max=60,\n"
- " default=5)\n");
- printf(" -t TRIGGERFILE trigger file to initiate failover (no default)\n");
- printf(" -V, --version output version information, then exit\n");
- printf(" -w MAXWAITTIME max seconds to wait for a file (0=no limit) (default=0)\n");
- printf(" -?, --help show this help, then exit\n");
- printf("\n"
- "Main intended use as restore_command in postgresql.conf:\n"
- " restore_command = 'pg_standby [OPTION]... ARCHIVELOCATION %%f %%p %%r'\n"
- "e.g.\n"
- " restore_command = 'pg_standby /mnt/server/archiverdir %%f %%p %%r'\n");
- printf("\nReport bugs to <%s>.\n", PACKAGE_BUGREPORT);
- printf("%s home page: <%s>\n", PACKAGE_NAME, PACKAGE_URL);
-}
-
-#ifndef WIN32
-static void
-sighandler(int sig)
-{
- signaled = true;
-}
-
-/* We don't want SIGQUIT to core dump */
-static void
-sigquit_handler(int sig)
-{
- pqsignal(SIGINT, SIG_DFL);
- kill(getpid(), SIGINT);
-}
-#endif
-
-/*------------ MAIN ----------------------------------------*/
-int
-main(int argc, char **argv)
-{
- int c;
-
- progname = get_progname(argv[0]);
-
- if (argc > 1)
- {
- if (strcmp(argv[1], "--help") == 0 || strcmp(argv[1], "-?") == 0)
- {
- usage();
- exit(0);
- }
- if (strcmp(argv[1], "--version") == 0 || strcmp(argv[1], "-V") == 0)
- {
- puts("pg_standby (PostgreSQL) " PG_VERSION);
- exit(0);
- }
- }
-
-#ifndef WIN32
-
- /*
- * You can send SIGUSR1 to trigger failover.
- *
- * Postmaster uses SIGQUIT to request immediate shutdown. The default
- * action is to core dump, but we don't want that, so trap it and commit
- * suicide without core dump.
- *
- * We used to use SIGINT and SIGQUIT to trigger failover, but that turned
- * out to be a bad idea because postmaster uses SIGQUIT to request
- * immediate shutdown. We still trap SIGINT, but that may change in a
- * future release.
- *
- * There's no way to trigger failover via signal on Windows.
- */
- (void) pqsignal(SIGUSR1, sighandler);
- (void) pqsignal(SIGINT, sighandler); /* deprecated, use SIGUSR1 */
- (void) pqsignal(SIGQUIT, sigquit_handler);
-#endif
-
- while ((c = getopt(argc, argv, "cdk:lr:s:t:w:")) != -1)
- {
- switch (c)
- {
- case 'c': /* Use copy */
- restoreCommandType = RESTORE_COMMAND_COPY;
- break;
- case 'd': /* Debug mode */
- debug = true;
- break;
- case 'k': /* keepfiles */
- keepfiles = atoi(optarg);
- if (keepfiles < 0)
- {
- fprintf(stderr, "%s: -k keepfiles must be >= 0\n", progname);
- exit(2);
- }
- break;
- case 'l': /* Use link */
-
- /*
- * Link feature disabled, possibly permanently. Linking causes
- * a problem after recovery ends that is not currently
- * resolved by PostgreSQL. 25 Jun 2009
- */
-#ifdef NOT_USED
- restoreCommandType = RESTORE_COMMAND_LINK;
-#endif
- break;
- case 'r': /* Retries */
- maxretries = atoi(optarg);
- if (maxretries < 0)
- {
- fprintf(stderr, "%s: -r maxretries must be >= 0\n", progname);
- exit(2);
- }
- break;
- case 's': /* Sleep time */
- sleeptime = atoi(optarg);
- if (sleeptime <= 0 || sleeptime > 60)
- {
- fprintf(stderr, "%s: -s sleeptime incorrectly set\n", progname);
- exit(2);
- }
- break;
- case 't': /* Trigger file */
- triggerPath = pg_strdup(optarg);
- break;
- case 'w': /* Max wait time */
- maxwaittime = atoi(optarg);
- if (maxwaittime < 0)
- {
- fprintf(stderr, "%s: -w maxwaittime incorrectly set\n", progname);
- exit(2);
- }
- break;
- default:
- fprintf(stderr, "Try \"%s --help\" for more information.\n", progname);
- exit(2);
- break;
- }
- }
-
- /*
- * Parameter checking - after checking to see if trigger file present
- */
- if (argc == 1)
- {
- fprintf(stderr, "%s: not enough command-line arguments\n", progname);
- exit(2);
- }
-
- /*
- * We will go to the archiveLocation to get nextWALFileName.
- * nextWALFileName may not exist yet, which would not be an error, so we
- * separate the archiveLocation and nextWALFileName so we can check
- * separately whether archiveLocation exists, if not that is an error
- */
- if (optind < argc)
- {
- archiveLocation = argv[optind];
- optind++;
- }
- else
- {
- fprintf(stderr, "%s: must specify archive location\n", progname);
- fprintf(stderr, "Try \"%s --help\" for more information.\n", progname);
- exit(2);
- }
-
- if (optind < argc)
- {
- nextWALFileName = argv[optind];
- optind++;
- }
- else
- {
- fprintf(stderr, "%s: must specify WAL file name as second non-option argument (use \"%%f\")\n", progname);
- fprintf(stderr, "Try \"%s --help\" for more information.\n", progname);
- exit(2);
- }
-
- if (optind < argc)
- {
- xlogFilePath = argv[optind];
- optind++;
- }
- else
- {
- fprintf(stderr, "%s: must specify xlog destination as third non-option argument (use \"%%p\")\n", progname);
- fprintf(stderr, "Try \"%s --help\" for more information.\n", progname);
- exit(2);
- }
-
- if (optind < argc)
- {
- restartWALFileName = argv[optind];
- optind++;
- }
-
- CustomizableInitialize();
-
- if (debug)
- {
- fprintf(stderr, "Trigger file: %s\n", triggerPath ? triggerPath : "<not set>");
- fprintf(stderr, "Waiting for WAL file: %s\n", nextWALFileName);
- fprintf(stderr, "WAL file path: %s\n", WALFilePath);
- fprintf(stderr, "Restoring to: %s\n", xlogFilePath);
- fprintf(stderr, "Sleep interval: %d second%s\n",
- sleeptime, (sleeptime > 1 ? "s" : " "));
- fprintf(stderr, "Max wait interval: %d %s\n",
- maxwaittime, (maxwaittime > 0 ? "seconds" : "forever"));
- fprintf(stderr, "Command for restore: %s\n", restoreCommand);
- fflush(stderr);
- }
-
- /*
- * Check for initial history file: always the first file to be requested
- * It's OK if the file isn't there - all other files need to wait
- */
- if (IsTLHistoryFileName(nextWALFileName))
- {
- nextWALFileType = XLOG_HISTORY;
- if (RestoreWALFileForRecovery())
- exit(0);
- else
- {
- if (debug)
- {
- fprintf(stderr, "history file not found\n");
- fflush(stderr);
- }
- exit(1);
- }
- }
-
- /*
- * Main wait loop
- */
- for (;;)
- {
- /* Check for trigger file or signal first */
- CheckForExternalTrigger();
-#ifndef WIN32
- if (signaled)
- {
- Failover = FastFailover;
- if (debug)
- {
- fprintf(stderr, "signaled to exit: fast failover\n");
- fflush(stderr);
- }
- }
-#endif
-
- /*
- * Check for fast failover immediately, before checking if the
- * requested WAL file is available
- */
- if (Failover == FastFailover)
- exit(1);
-
- if (CustomizableNextWALFileReady())
- {
- /*
- * Once we have restored this file successfully we can remove some
- * prior WAL files. If this restore fails we mustn't remove any
- * file because some of them will be requested again immediately
- * after the failed restore, or when we restart recovery.
- */
- if (RestoreWALFileForRecovery())
- {
- if (need_cleanup)
- CustomizableCleanupPriorWALFiles();
-
- exit(0);
- }
- else
- {
- /* Something went wrong in copying the file */
- exit(1);
- }
- }
-
- /* Check for smart failover if the next WAL file was not available */
- if (Failover == SmartFailover)
- exit(1);
-
- if (sleeptime <= 60)
- pg_usleep(sleeptime * 1000000L);
-
- waittime += sleeptime;
- if (waittime >= maxwaittime && maxwaittime > 0)
- {
- Failover = FastFailover;
- if (debug)
- {
- fprintf(stderr, "Timed out after %d seconds: fast failover\n",
- waittime);
- fflush(stderr);
- }
- }
- if (debug)
- {
- fprintf(stderr, "WAL file not present yet.");
- if (triggerPath)
- fprintf(stderr, " Checking for trigger file...");
- fprintf(stderr, "\n");
- fflush(stderr);
- }
- }
-}
diff --git a/doc/src/sgml/contrib.sgml b/doc/src/sgml/contrib.sgml
index ae2759be55..d3ca4b6932 100644
--- a/doc/src/sgml/contrib.sgml
+++ b/doc/src/sgml/contrib.sgml
@@ -192,13 +192,12 @@ pages.
<title>Server Applications</title>
<para>
- This section covers <productname>PostgreSQL</productname> server-related
- applications in <literal>contrib</literal>. They are typically run on the
- host where the database server resides. See also <xref
+ Some applications run on the <productname>PostgreSQL</productname> server
+ itself. Currently, no such applications are included in the
+ <literal>contrib</literal> directory. See also <xref
linkend="reference-server"/> for information about server applications that
are part of the core <productname>PostgreSQL</productname> distribution.
</para>
- &pgstandby;
</sect1>
</appendix>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 38e8aa0bbf..db1d369743 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -138,7 +138,6 @@
<!ENTITY pgfreespacemap SYSTEM "pgfreespacemap.sgml">
<!ENTITY pgprewarm SYSTEM "pgprewarm.sgml">
<!ENTITY pgrowlocks SYSTEM "pgrowlocks.sgml">
-<!ENTITY pgstandby SYSTEM "pgstandby.sgml">
<!ENTITY pgstatstatements SYSTEM "pgstatstatements.sgml">
<!ENTITY pgstattuple SYSTEM "pgstattuple.sgml">
<!ENTITY pgsurgery SYSTEM "pgsurgery.sgml">
diff --git a/doc/src/sgml/high-availability.sgml b/doc/src/sgml/high-availability.sgml
index dc263e4106..9364dc74f7 100644
--- a/doc/src/sgml/high-availability.sgml
+++ b/doc/src/sgml/high-availability.sgml
@@ -699,11 +699,9 @@ protocol to make nodes agree on a serializable transactional order.
<note>
<para>
- Do not use pg_standby or similar tools with the built-in standby mode
- described here. <xref linkend="guc-restore-command"/> should return immediately
+ <xref linkend="guc-restore-command"/> should return immediately
if the file does not exist; the server will retry the command again if
- necessary. See <xref linkend="log-shipping-alternative"/>
- for using tools like pg_standby.
+ necessary.
</para>
</note>
@@ -1494,8 +1492,7 @@ synchronous_standby_names = 'ANY 2 (s1, s2, s3)'
<para>
An alternative to the built-in standby mode described in the previous
sections is to use a <varname>restore_command</varname> that polls the archive location.
- This was the only option available in versions 8.4 and below. See the
- <xref linkend="pgstandby"/> module for a reference implementation of this.
+ This was the only option available in versions 8.4 and below.
</para>
<para>
@@ -1551,14 +1548,6 @@ if (!triggered)
</programlisting>
</para>
- <para>
- A working example of a waiting <varname>restore_command</varname> is provided
- in the <xref linkend="pgstandby"/> module. It
- should be used as a reference on how to correctly implement the logic
- described above. It can also be extended as needed to support specific
- configurations and environments.
- </para>
-
<para>
The method for triggering failover is an important part of planning
and design. One potential option is the <varname>restore_command</varname>
diff --git a/doc/src/sgml/pgstandby.sgml b/doc/src/sgml/pgstandby.sgml
deleted file mode 100644
index 66a6255930..0000000000
--- a/doc/src/sgml/pgstandby.sgml
+++ /dev/null
@@ -1,394 +0,0 @@
-<!-- doc/src/sgml/pgstandby.sgml -->
-
-<refentry id="pgstandby">
- <indexterm zone="pgstandby">
- <primary>pg_standby</primary>
- </indexterm>
-
- <refmeta>
- <refentrytitle><application>pg_standby</application></refentrytitle>
- <manvolnum>1</manvolnum>
- <refmiscinfo>Application</refmiscinfo>
- </refmeta>
-
- <refnamediv>
- <refname>pg_standby</refname>
- <refpurpose>supports the creation of a <productname>PostgreSQL</productname> warm standby server</refpurpose>
- </refnamediv>
-
- <refsynopsisdiv>
- <cmdsynopsis>
- <command>pg_standby</command>
- <arg rep="repeat"><replaceable>option</replaceable></arg>
- <arg choice="plain"><replaceable>archivelocation</replaceable></arg>
- <arg choice="plain"><replaceable>nextwalfile</replaceable></arg>
- <arg choice="plain"><replaceable>walfilepath</replaceable></arg>
- <arg choice="opt"><replaceable>restartwalfile</replaceable></arg>
- </cmdsynopsis>
- </refsynopsisdiv>
-
- <refsect1>
- <title>Description</title>
-
- <para>
- <application>pg_standby</application> supports creation of a <quote>warm standby</quote>
- database server. It is designed to be a production-ready program, as well
- as a customizable template should you require specific modifications.
- </para>
-
- <para>
- <application>pg_standby</application> is designed to be a waiting
- <varname>restore_command</varname>, which is needed to turn a standard
- archive recovery into a warm standby operation. Other
- configuration is required as well, all of which is described in the main
- server manual (see <xref linkend="warm-standby"/>).
- </para>
-
- <para>
- To configure a standby
- server to use <application>pg_standby</application>, put this into its
- <filename>postgresql.conf</filename> configuration file:
-<programlisting>
-restore_command = 'pg_standby <replaceable>archiveDir</replaceable> %f %p %r'
-</programlisting>
- where <replaceable>archiveDir</replaceable> is the directory from which WAL segment
- files should be restored.
- </para>
- <para>
- If <replaceable>restartwalfile</replaceable> is specified, normally by using the
- <literal>%r</literal> macro, then all WAL files logically preceding this
- file will be removed from <replaceable>archivelocation</replaceable>. This minimizes
- the number of files that need to be retained, while preserving
- crash-restart capability. Use of this parameter is appropriate if the
- <replaceable>archivelocation</replaceable> is a transient staging area for this
- particular standby server, but <emphasis>not</emphasis> when the
- <replaceable>archivelocation</replaceable> is intended as a long-term WAL archive area.
- </para>
- <para>
- <application>pg_standby</application> assumes that
- <replaceable>archivelocation</replaceable> is a directory readable by the
- server-owning user. If <replaceable>restartwalfile</replaceable> (or <literal>-k</literal>)
- is specified,
- the <replaceable>archivelocation</replaceable> directory must be writable too.
- </para>
- <para>
- There are two ways to fail over to a <quote>warm standby</quote> database server
- when the primary server fails:
-
- <variablelist>
- <varlistentry>
- <term>Smart Failover</term>
- <listitem>
- <para>
- In smart failover, the server is brought up after applying all WAL
- files available in the archive. This results in zero data loss, even if
- the standby server has fallen behind, but if there is a lot of
- unapplied WAL it can be a long time before the standby server becomes
- ready. To trigger a smart failover, create a trigger file containing
- the word <literal>smart</literal>, or just create it and leave it empty.
- </para>
- </listitem>
- </varlistentry>
- <varlistentry>
- <term>Fast Failover</term>
- <listitem>
- <para>
- In fast failover, the server is brought up immediately. Any WAL files
- in the archive that have not yet been applied will be ignored, and
- all transactions in those files are lost. To trigger a fast failover,
- create a trigger file and write the word <literal>fast</literal> into it.
- <application>pg_standby</application> can also be configured to execute a fast
- failover automatically if no new WAL file appears within a defined
- interval.
- </para>
- </listitem>
- </varlistentry>
- </variablelist>
- </para>
-
- </refsect1>
-
- <refsect1>
- <title>Options</title>
-
- <para>
- <application>pg_standby</application> accepts the following command-line arguments:
-
- <variablelist>
-
- <varlistentry>
- <term><option>-c</option></term>
- <listitem>
- <para>
- Use <literal>cp</literal> or <literal>copy</literal> command to restore WAL files
- from archive. This is the only supported behavior so this option is useless.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-d</option></term>
- <listitem>
- <para>
- Print lots of debug logging output on <filename>stderr</filename>.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-k</option></term>
- <listitem>
- <para>
- Remove files from <replaceable>archivelocation</replaceable> so that
- no more than this many WAL files before the current one are kept in the
- archive. Zero (the default) means not to remove any files from
- <replaceable>archivelocation</replaceable>.
- This parameter will be silently ignored if
- <replaceable>restartwalfile</replaceable> is specified, since that
- specification method is more accurate in determining the correct
- archive cut-off point.
- Use of this parameter is <emphasis>deprecated</emphasis> as of
- <productname>PostgreSQL</productname> 8.3; it is safer and more efficient to
- specify a <replaceable>restartwalfile</replaceable> parameter. A too
- small setting could result in removal of files that are still needed
- for a restart of the standby server, while a too large setting wastes
- archive space.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-r</option> <replaceable>maxretries</replaceable></term>
- <listitem>
- <para>
- Set the maximum number of times to retry the copy command if
- it fails (default 3). After each failure, we wait for
- <replaceable>sleeptime</replaceable> * <replaceable>num_retries</replaceable>
- so that the wait time increases progressively. So by default,
- we will wait 5 secs, 10 secs, then 15 secs before reporting
- the failure back to the standby server. This will be
- interpreted as end of recovery and the standby will come
- up fully as a result.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-s</option> <replaceable>sleeptime</replaceable></term>
- <listitem>
- <para>
- Set the number of seconds (up to 60, default 5) to sleep between
- tests to see if the WAL file to be restored is available in
- the archive yet. The default setting is not necessarily
- recommended; consult <xref linkend="warm-standby"/> for discussion.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-t</option> <replaceable>triggerfile</replaceable></term>
- <listitem>
- <para>
- Specify a trigger file whose presence should cause failover.
- It is recommended that you use a structured file name to
- avoid confusion as to which server is being triggered
- when multiple servers exist on the same system; for example
- <filename>/tmp/pgsql.trigger.5432</filename>.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-V</option></term>
- <term><option>--version</option></term>
- <listitem>
- <para>
- Print the <application>pg_standby</application> version and exit.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-w</option> <replaceable>maxwaittime</replaceable></term>
- <listitem>
- <para>
- Set the maximum number of seconds to wait for the next WAL file,
- after which a fast failover will be performed.
- A setting of zero (the default) means wait forever.
- The default setting is not necessarily recommended;
- consult <xref linkend="warm-standby"/> for discussion.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-?</option></term>
- <term><option>--help</option></term>
- <listitem>
- <para>
- Show help about <application>pg_standby</application> command line
- arguments, and exit.
- </para>
- </listitem>
- </varlistentry>
- </variablelist>
- </para>
-
- </refsect1>
-
- <refsect1>
- <title>Notes</title>
-
- <para>
- <application>pg_standby</application> is designed to work with
- <productname>PostgreSQL</productname> 8.2 and later.
- </para>
- <para>
- <productname>PostgreSQL</productname> 8.3 provides the <literal>%r</literal> macro,
- which is designed to let <application>pg_standby</application> know the
- last file it needs to keep. With <productname>PostgreSQL</productname> 8.2, the
- <literal>-k</literal> option must be used if archive cleanup is
- required. This option remains available in 8.3, but its use is deprecated.
- </para>
- <para>
- <productname>PostgreSQL</productname> 8.4 provides the
- <varname>recovery_end_command</varname> option. Without this option
- a leftover trigger file can be hazardous.
- </para>
-
- <para>
- <application>pg_standby</application> is written in C and has an
- easy-to-modify source code, with specifically designated sections to modify
- for your own needs
- </para>
- </refsect1>
-
- <refsect1>
- <title>Examples</title>
-
- <para>On Linux or Unix systems, you might use:
-
-<programlisting>
-archive_command = 'cp %p .../archive/%f'
-
-restore_command = 'pg_standby -d -s 2 -t /tmp/pgsql.trigger.5442 .../archive %f %p %r 2>>standby.log'
-
-recovery_end_command = 'rm -f /tmp/pgsql.trigger.5442'
-</programlisting>
- where the archive directory is physically located on the standby server,
- so that the <varname>archive_command</varname> is accessing it across NFS,
- but the files are local to the standby (enabling use of <literal>ln</literal>).
- This will:
- <itemizedlist>
- <listitem>
- <para>
- produce debugging output in <filename>standby.log</filename>
- </para>
- </listitem>
- <listitem>
- <para>
- sleep for 2 seconds between checks for next WAL file availability
- </para>
- </listitem>
- <listitem>
- <para>
- stop waiting only when a trigger file called
- <filename>/tmp/pgsql.trigger.5442</filename> appears,
- and perform failover according to its content
- </para>
- </listitem>
- <listitem>
- <para>
- remove the trigger file when recovery ends
- </para>
- </listitem>
- <listitem>
- <para>
- remove no-longer-needed files from the archive directory
- </para>
- </listitem>
- </itemizedlist>
- </para>
-
- <para>On Windows, you might use:
-
-<programlisting>
-archive_command = 'copy %p ...\\archive\\%f'
-
-restore_command = 'pg_standby -d -s 5 -t C:\pgsql.trigger.5442 ...\archive %f %p %r 2>>standby.log'
-
-recovery_end_command = 'del C:\pgsql.trigger.5442'
-</programlisting>
- Note that backslashes need to be doubled in the
- <varname>archive_command</varname>, but <emphasis>not</emphasis> in the
- <varname>restore_command</varname> or <varname>recovery_end_command</varname>.
- This will:
- <itemizedlist>
- <listitem>
- <para>
- use the <literal>copy</literal> command to restore WAL files from archive
- </para>
- </listitem>
- <listitem>
- <para>
- produce debugging output in <filename>standby.log</filename>
- </para>
- </listitem>
- <listitem>
- <para>
- sleep for 5 seconds between checks for next WAL file availability
- </para>
- </listitem>
- <listitem>
- <para>
- stop waiting only when a trigger file called
- <filename>C:\pgsql.trigger.5442</filename> appears,
- and perform failover according to its content
- </para>
- </listitem>
- <listitem>
- <para>
- remove the trigger file when recovery ends
- </para>
- </listitem>
- <listitem>
- <para>
- remove no-longer-needed files from the archive directory
- </para>
- </listitem>
- </itemizedlist>
- </para>
-
- <para>
- The <literal>copy</literal> command on Windows sets the final file size
- before the file is completely copied, which would ordinarily confuse
- <application>pg_standby</application>. Therefore
- <application>pg_standby</application> waits <replaceable>sleeptime</replaceable>
- seconds once it sees the proper file size. GNUWin32's <literal>cp</literal>
- sets the file size only after the file copy is complete.
- </para>
-
- <para>
- Since the Windows example uses <literal>copy</literal> at both ends, either
- or both servers might be accessing the archive directory across the
- network.
- </para>
-
- </refsect1>
-
- <refsect1>
- <title>Author</title>
-
- <para>
- Simon Riggs <email>simon@2ndquadrant.com</email>
- </para>
- </refsect1>
-
- <refsect1>
- <title>See Also</title>
-
- <simplelist type="inline">
- <member><xref linkend="pgarchivecleanup"/></member>
- </simplelist>
- </refsect1>
-</refentry>
diff --git a/doc/src/sgml/ref/pgarchivecleanup.sgml b/doc/src/sgml/ref/pgarchivecleanup.sgml
index 56f02fc0e6..e27db3c077 100644
--- a/doc/src/sgml/ref/pgarchivecleanup.sgml
+++ b/doc/src/sgml/ref/pgarchivecleanup.sgml
@@ -205,11 +205,4 @@ archive_cleanup_command = 'pg_archivecleanup -d /mnt/standby/archive %r 2>>clean
</itemizedlist>
</refsect1>
- <refsect1>
- <title>See Also</title>
-
- <simplelist type="inline">
- <member><xref linkend="pgstandby"/></member>
- </simplelist>
- </refsect1>
</refentry>
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index cc007b8963..3cc7b9f6d0 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -4199,7 +4199,7 @@ RemoveXlogFile(const char *segname, XLogSegNo recycleSegNo,
/*
* Before deleting the file, see if it can be recycled as a future log
- * segment. Only recycle normal files, pg_standby for example can create
+ * segment. Only recycle normal files, because we don't want to recycle
* symbolic links pointing to a separate archive directory.
*/
if (wal_recycle &&
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 7213e65e08..90328db04e 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -34,8 +34,8 @@ my @unlink_on_exit;
# Set of variables for modules in contrib/ and src/test/modules/
my $contrib_defines = { 'refint' => 'REFINT_VERBOSE' };
my @contrib_uselibpq = ('dblink', 'oid2name', 'postgres_fdw', 'vacuumlo');
-my @contrib_uselibpgport = ('oid2name', 'pg_standby', 'vacuumlo');
-my @contrib_uselibpgcommon = ('oid2name', 'pg_standby', 'vacuumlo');
+my @contrib_uselibpgport = ('oid2name', 'vacuumlo');
+my @contrib_uselibpgcommon = ('oid2name', 'vacuumlo');
my $contrib_extralibs = undef;
my $contrib_extraincludes = { 'dblink' => ['src/backend'] };
my $contrib_extrasource = {
--
2.20.1
v4-0002-Remove-documentation-of-waiting-restore_command.patch (text/x-patch, 6.9K) download
From c2b957cb0f7e62c2a9be75fc18eed6afb119d12a Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.munro@gmail.com>
Date: Wed, 27 Jan 2021 16:04:55 +1300
Subject: [PATCH v4 2/2] Remove documentation of waiting restore_command.
Following the removal of pg_standby, also remove the documentation
section that describes how to write your own "waiting restore_command".
Such restore commands do not interact well with a proposed recovery
prefetching feature.
Discussion: https://postgr.es/m/20201029024412.GP5380%40telsasoft.com
Reviewed-by: Michael Paquier <michael@paquier.xyz>
---
doc/src/sgml/high-availability.sgml | 137 ----------------------------
1 file changed, 137 deletions(-)
diff --git a/doc/src/sgml/high-availability.sgml b/doc/src/sgml/high-availability.sgml
index 9364dc74f7..f49f5c0108 100644
--- a/doc/src/sgml/high-availability.sgml
+++ b/doc/src/sgml/high-availability.sgml
@@ -1486,143 +1486,6 @@ synchronous_standby_names = 'ANY 2 (s1, s2, s3)'
</para>
</sect1>
- <sect1 id="log-shipping-alternative">
- <title>Alternative Method for Log Shipping</title>
-
- <para>
- An alternative to the built-in standby mode described in the previous
- sections is to use a <varname>restore_command</varname> that polls the archive location.
- This was the only option available in versions 8.4 and below.
- </para>
-
- <para>
- Note that in this mode, the server will apply WAL one file at a
- time, so if you use the standby server for queries (see Hot Standby),
- there is a delay between an action in the primary and when the
- action becomes visible in the standby, corresponding to the time it takes
- to fill up the WAL file. <varname>archive_timeout</varname> can be used to make that delay
- shorter. Also note that you can't combine streaming replication with
- this method.
- </para>
-
- <para>
- The operations that occur on both primary and standby servers are
- normal continuous archiving and recovery tasks. The only point of
- contact between the two database servers is the archive of WAL files
- that both share: primary writing to the archive, standby reading from
- the archive. Care must be taken to ensure that WAL archives from separate
- primary servers do not become mixed together or confused. The archive
- need not be large if it is only required for standby operation.
- </para>
-
- <para>
- The magic that makes the two loosely coupled servers work together is
- simply a <varname>restore_command</varname> used on the standby that,
- when asked for the next WAL file, waits for it to become available from
- the primary. Normal recovery
- processing would request a file from the WAL archive, reporting failure
- if the file was unavailable. For standby processing it is normal for
- the next WAL file to be unavailable, so the standby must wait for
- it to appear. For files ending in
- <literal>.history</literal> there is no need to wait, and a non-zero return
- code must be returned. A waiting <varname>restore_command</varname> can be
- written as a custom script that loops after polling for the existence of
- the next WAL file. There must also be some way to trigger failover, which
- should interrupt the <varname>restore_command</varname>, break the loop and
- return a file-not-found error to the standby server. This ends recovery
- and the standby will then come up as a normal server.
- </para>
-
- <para>
- Pseudocode for a suitable <varname>restore_command</varname> is:
-<programlisting>
-triggered = false;
-while (!NextWALFileReady() && !triggered)
-{
- sleep(100000L); /* wait for ~0.1 sec */
- if (CheckForExternalTrigger())
- triggered = true;
-}
-if (!triggered)
- CopyWALFileForRecovery();
-</programlisting>
- </para>
-
- <para>
- The method for triggering failover is an important part of planning
- and design. One potential option is the <varname>restore_command</varname>
- command. It is executed once for each WAL file, but the process
- running the <varname>restore_command</varname> is created and dies for
- each file, so there is no daemon or server process, and
- signals or a signal handler cannot be used. Therefore, the
- <varname>restore_command</varname> is not suitable to trigger failover.
- It is possible to use a simple timeout facility, especially if
- used in conjunction with a known <varname>archive_timeout</varname>
- setting on the primary. However, this is somewhat error prone
- since a network problem or busy primary server might be sufficient
- to initiate failover. A notification mechanism such as the explicit
- creation of a trigger file is ideal, if this can be arranged.
- </para>
-
- <sect2 id="warm-standby-config">
- <title>Implementation</title>
-
- <para>
- The short procedure for configuring a standby server using this alternative
- method is as follows. For
- full details of each step, refer to previous sections as noted.
- <orderedlist>
- <listitem>
- <para>
- Set up primary and standby systems as nearly identical as
- possible, including two identical copies of
- <productname>PostgreSQL</productname> at the same release level.
- </para>
- </listitem>
- <listitem>
- <para>
- Set up continuous archiving from the primary to a WAL archive
- directory on the standby server. Ensure that
- <xref linkend="guc-archive-mode"/>,
- <xref linkend="guc-archive-command"/> and
- <xref linkend="guc-archive-timeout"/>
- are set appropriately on the primary
- (see <xref linkend="backup-archiving-wal"/>).
- </para>
- </listitem>
- <listitem>
- <para>
- Make a base backup of the primary server (see <xref
- linkend="backup-base-backup"/>), and load this data onto the standby.
- </para>
- </listitem>
- <listitem>
- <para>
- Begin recovery on the standby server from the local WAL
- archive, using <varname>restore_command</varname> that waits
- as described previously (see <xref linkend="backup-pitr-recovery"/>).
- </para>
- </listitem>
- </orderedlist>
- </para>
-
- <para>
- Recovery treats the WAL archive as read-only, so once a WAL file has
- been copied to the standby system it can be copied to tape at the same
- time as it is being read by the standby database server.
- Thus, running a standby server for high availability can be performed at
- the same time as files are stored for longer term disaster recovery
- purposes.
- </para>
-
- <para>
- For testing purposes, it is possible to run both primary and standby
- servers on the same system. This does not provide any worthwhile
- improvement in server robustness, nor would it be described as HA.
- </para>
- </sect2>
- </sect1>
-
<sect1 id="hot-standby">
<title>Hot Standby</title>
--
2.20.1
* Re: [PATCH] remove pg_standby
2020-11-21 19:41 Re: [PATCH] remove pg_standby Justin Pryzby <pryzby@telsasoft.com>
2020-11-25 09:04 ` Re: [PATCH] remove pg_standby Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2021-01-27 03:13 ` Re: [PATCH] remove pg_standby Thomas Munro <thomas.munro@gmail.com>
2021-01-27 05:06 ` Re: [PATCH] remove pg_standby Michael Paquier <michael@paquier.xyz>
2021-01-27 05:32 ` Re: [PATCH] remove pg_standby Thomas Munro <thomas.munro@gmail.com>
@ 2021-01-27 08:08 ` Fujii Masao <masao.fujii@oss.nttdata.com>
2021-01-28 07:36 ` Re: [PATCH] remove pg_standby Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 25+ messages in thread
From: Fujii Masao @ 2021-01-27 08:08 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; Michael Paquier <michael@paquier.xyz>; +Cc: Peter Eisentraut <peter.eisentraut@enterprisedb.com>; Justin Pryzby <pryzby@telsasoft.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Tom Lane <tgl@sss.pgh.pa.us>; Stephen Frost <sfrost@snowman.net>; Ian Lawrence Barwick <barwick@gmail.com>; pgsql-hackers
On 2021/01/27 14:32, Thomas Munro wrote:
> On Wed, Jan 27, 2021 at 6:06 PM Michael Paquier <michael@paquier.xyz> wrote:
>> On Wed, Jan 27, 2021 at 04:13:24PM +1300, Thomas Munro wrote:
>>> I would like to commit this, because "waiting restore commands" have
>>> confusing interactions with my proposed prefetching-during-recovery
>>> patch[1]. Here's a version that fixes an error when building the docs
>>> (there was a stray remaining <xref linkend="pgstandby"/>), and adds a
>>> commit message. Any objections?
I agree with this direction (i.e, remove pg_standby). BTW last month when I gave the talk about possible retire of pg_standby at PostgreSQL Unconference Tokyo, no one in audience complained about that retire.
But one question is; shouldn't we follow "usual" way to retire the feature instead of dropping that immediately? That is, mark pg_standby as obsolete, announce that pg_standby will be dropped after several releases, and then drop pg_standby. This seems safe because there might be some users. While it's been marked as obsolete, maybe WAL prefetch feature doesn't work with pg_standby, but we can live with that because it's obsolete.
Regards,
--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: [PATCH] remove pg_standby
2020-11-21 19:41 Re: [PATCH] remove pg_standby Justin Pryzby <pryzby@telsasoft.com>
2020-11-25 09:04 ` Re: [PATCH] remove pg_standby Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2021-01-27 03:13 ` Re: [PATCH] remove pg_standby Thomas Munro <thomas.munro@gmail.com>
2021-01-27 05:06 ` Re: [PATCH] remove pg_standby Michael Paquier <michael@paquier.xyz>
2021-01-27 05:32 ` Re: [PATCH] remove pg_standby Thomas Munro <thomas.munro@gmail.com>
2021-01-27 08:08 ` Re: [PATCH] remove pg_standby Fujii Masao <masao.fujii@oss.nttdata.com>
@ 2021-01-28 07:36 ` Michael Paquier <michael@paquier.xyz>
2021-01-28 22:13 ` Re: [PATCH] remove pg_standby Thomas Munro <thomas.munro@gmail.com>
0 siblings, 1 reply; 25+ messages in thread
From: Michael Paquier @ 2021-01-28 07:36 UTC (permalink / raw)
To: Fujii Masao <masao.fujii@oss.nttdata.com>; +Cc: Thomas Munro <thomas.munro@gmail.com>; Peter Eisentraut <peter.eisentraut@enterprisedb.com>; Justin Pryzby <pryzby@telsasoft.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Tom Lane <tgl@sss.pgh.pa.us>; Stephen Frost <sfrost@snowman.net>; Ian Lawrence Barwick <barwick@gmail.com>; pgsql-hackers
On Wed, Jan 27, 2021 at 05:08:56PM +0900, Fujii Masao wrote:
> But one question is; shouldn't we follow "usual" way to retire the
> feature instead of dropping that immediately? That is, mark
> pg_standby as obsolete, announce that pg_standby will be dropped
> after several releases, and then drop pg_standby. This seems safe
> because there might be some users. While it's been marked as
> obsolete, maybe WAL prefetch feature doesn't work with pg_standby,
> but we can live with that because it's obsolete.
Thanks. FWIW, at this stage, my take is just to move on and remove
it. If we mark that as obsolete, it will stay around forever while
annoying future development.
--
Michael
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: [PATCH] remove pg_standby
2020-11-21 19:41 Re: [PATCH] remove pg_standby Justin Pryzby <pryzby@telsasoft.com>
2020-11-25 09:04 ` Re: [PATCH] remove pg_standby Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2021-01-27 03:13 ` Re: [PATCH] remove pg_standby Thomas Munro <thomas.munro@gmail.com>
2021-01-27 05:06 ` Re: [PATCH] remove pg_standby Michael Paquier <michael@paquier.xyz>
2021-01-27 05:32 ` Re: [PATCH] remove pg_standby Thomas Munro <thomas.munro@gmail.com>
2021-01-27 08:08 ` Re: [PATCH] remove pg_standby Fujii Masao <masao.fujii@oss.nttdata.com>
2021-01-28 07:36 ` Re: [PATCH] remove pg_standby Michael Paquier <michael@paquier.xyz>
@ 2021-01-28 22:13 ` Thomas Munro <thomas.munro@gmail.com>
2021-01-29 01:19 ` Re: [PATCH] remove pg_standby Thomas Munro <thomas.munro@gmail.com>
0 siblings, 1 reply; 25+ messages in thread
From: Thomas Munro @ 2021-01-28 22:13 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: Fujii Masao <masao.fujii@oss.nttdata.com>; Peter Eisentraut <peter.eisentraut@enterprisedb.com>; Justin Pryzby <pryzby@telsasoft.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Tom Lane <tgl@sss.pgh.pa.us>; Stephen Frost <sfrost@snowman.net>; Ian Lawrence Barwick <barwick@gmail.com>; pgsql-hackers
On Thu, Jan 28, 2021 at 8:36 PM Michael Paquier <michael@paquier.xyz> wrote:
> On Wed, Jan 27, 2021 at 05:08:56PM +0900, Fujii Masao wrote:
> > But one question is; shouldn't we follow "usual" way to retire the
> > feature instead of dropping that immediately? That is, mark
> > pg_standby as obsolete, announce that pg_standby will be dropped
> > after several releases, and then drop pg_standby. This seems safe
> > because there might be some users. While it's been marked as
> > obsolete, maybe WAL prefetch feature doesn't work with pg_standby,
> > but we can live with that because it's obsolete.
>
> Thanks. FWIW, at this stage, my take is just to move on and remove
> it. If we mark that as obsolete, it will stay around forever while
> annoying future development.
I agree. Also, this thing is entirely separate from the server, so a
hypothetical user who really wants to upgrade to 14 but keep using
pg_standby a bit longer could always use the version that shipped with
13.
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: [PATCH] remove pg_standby
2020-11-21 19:41 Re: [PATCH] remove pg_standby Justin Pryzby <pryzby@telsasoft.com>
2020-11-25 09:04 ` Re: [PATCH] remove pg_standby Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2021-01-27 03:13 ` Re: [PATCH] remove pg_standby Thomas Munro <thomas.munro@gmail.com>
2021-01-27 05:06 ` Re: [PATCH] remove pg_standby Michael Paquier <michael@paquier.xyz>
2021-01-27 05:32 ` Re: [PATCH] remove pg_standby Thomas Munro <thomas.munro@gmail.com>
2021-01-27 08:08 ` Re: [PATCH] remove pg_standby Fujii Masao <masao.fujii@oss.nttdata.com>
2021-01-28 07:36 ` Re: [PATCH] remove pg_standby Michael Paquier <michael@paquier.xyz>
2021-01-28 22:13 ` Re: [PATCH] remove pg_standby Thomas Munro <thomas.munro@gmail.com>
@ 2021-01-29 01:19 ` Thomas Munro <thomas.munro@gmail.com>
0 siblings, 0 replies; 25+ messages in thread
From: Thomas Munro @ 2021-01-29 01:19 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: Fujii Masao <masao.fujii@oss.nttdata.com>; Peter Eisentraut <peter.eisentraut@enterprisedb.com>; Justin Pryzby <pryzby@telsasoft.com>; Heikki Linnakangas <hlinnaka@iki.fi>; Tom Lane <tgl@sss.pgh.pa.us>; Stephen Frost <sfrost@snowman.net>; Ian Lawrence Barwick <barwick@gmail.com>; pgsql-hackers
On Fri, Jan 29, 2021 at 11:13 AM Thomas Munro <thomas.munro@gmail.com> wrote:
> On Thu, Jan 28, 2021 at 8:36 PM Michael Paquier <michael@paquier.xyz> wrote:
> > On Wed, Jan 27, 2021 at 05:08:56PM +0900, Fujii Masao wrote:
> > > But one question is; shouldn't we follow "usual" way to retire the
> > > feature instead of dropping that immediately? That is, mark
> > > pg_standby as obsolete, announce that pg_standby will be dropped
> > > after several releases, and then drop pg_standby. This seems safe
> > > because there might be some users. While it's been marked as
> > > obsolete, maybe WAL prefetch feature doesn't work with pg_standby,
> > > but we can live with that because it's obsolete.
> >
> > Thanks. FWIW, at this stage, my take is just to move on and remove
> > it. If we mark that as obsolete, it will stay around forever while
> > annoying future development.
>
> I agree. Also, this thing is entirely separate from the server, so a
> hypothetical user who really wants to upgrade to 14 but keep using
> pg_standby a bit longer could always use the version that shipped with
> 13.
And, pushed.
^ permalink raw reply [nested|flat] 25+ messages in thread
end of thread, other threads:[~2021-01-29 01:19 UTC | newest]
Thread overview: 25+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2014-11-04 21:36 remove pg_standby? Peter Eisentraut <peter_e@gmx.net>
2014-11-08 05:01 ` Michael Paquier <michael.paquier@gmail.com>
2014-11-10 15:19 ` Magnus Hagander <magnus@hagander.net>
2014-11-10 17:16 ` Fujii Masao <masao.fujii@gmail.com>
2014-11-10 17:19 ` Magnus Hagander <magnus@hagander.net>
2014-11-10 17:43 ` Fujii Masao <masao.fujii@gmail.com>
2014-11-10 17:50 ` Joshua D. Drake <jd@commandprompt.com>
2014-11-10 18:48 ` Heikki Linnakangas <hlinnakangas@vmware.com>
2014-11-10 18:54 ` Magnus Hagander <magnus@hagander.net>
2015-02-27 20:00 ` Josh Berkus <josh@agliodbs.com>
2015-03-02 09:53 ` Fujii Masao <masao.fujii@gmail.com>
2015-03-02 11:37 ` Heikki Linnakangas <hlinnaka@iki.fi>
2015-03-02 12:04 ` Fujii Masao <masao.fujii@gmail.com>
2015-03-02 18:07 ` Josh Berkus <josh@agliodbs.com>
2015-03-02 23:22 ` Fujii Masao <masao.fujii@gmail.com>
2015-03-03 22:01 ` Robert Haas <robertmhaas@gmail.com>
2020-11-21 19:41 Re: [PATCH] remove pg_standby Justin Pryzby <pryzby@telsasoft.com>
2020-11-25 09:04 ` Re: [PATCH] remove pg_standby Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2021-01-27 03:13 ` Re: [PATCH] remove pg_standby Thomas Munro <thomas.munro@gmail.com>
2021-01-27 05:06 ` Re: [PATCH] remove pg_standby Michael Paquier <michael@paquier.xyz>
2021-01-27 05:32 ` Re: [PATCH] remove pg_standby Thomas Munro <thomas.munro@gmail.com>
2021-01-27 08:08 ` Re: [PATCH] remove pg_standby Fujii Masao <masao.fujii@oss.nttdata.com>
2021-01-28 07:36 ` Re: [PATCH] remove pg_standby Michael Paquier <michael@paquier.xyz>
2021-01-28 22:13 ` Re: [PATCH] remove pg_standby Thomas Munro <thomas.munro@gmail.com>
2021-01-29 01:19 ` Re: [PATCH] remove pg_standby Thomas Munro <thomas.munro@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox