pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedReq. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment"
5+ messages / 3 participants
[nested] [flat]
* Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment"
@ 2016-12-05 18:57 Tharmarajah, Sam <sambavan.tharmarajah@usask.ca>
2016-12-05 19:52 ` Re: Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment" John Scalia <jayknowsunix@gmail.com>
0 siblings, 1 reply; 5+ messages in thread
From: Tharmarajah, Sam @ 2016-12-05 18:57 UTC (permalink / raw)
To: pgsql-admin
Hi
I am a newbie to postgres. We are on postgres 9.1 running on RedHat 6. When we do the postgres backups we are noticing the following errors
pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment 000000010000020B00000040 has already been removed
But when I look into pg_xlog, I notice
000000010000020B00000040.00000118.backup
May I know how to resolve this error? Are the backups done with these errors, recoverable?
Thanks in advance for your support.
Thanx,
Sam
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment"
2016-12-05 18:57 Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment" Tharmarajah, Sam <sambavan.tharmarajah@usask.ca>
@ 2016-12-05 19:52 ` John Scalia <jayknowsunix@gmail.com>
2016-12-05 20:27 ` Re: Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment" Tharmarajah, Sam <sambavan.tharmarajah@usask.ca>
0 siblings, 1 reply; 5+ messages in thread
From: John Scalia @ 2016-12-05 19:52 UTC (permalink / raw)
To: Tharmarajah, Sam <sambavan.tharmarajah@usask.ca>; +Cc: pgsql-admin
More than likely your keep_wal_segments setting is too low. How long is the pg_basebackup taking? Remember that segments are generated about every 5 minutes, so if the backup takes an hour, you need at least 12 segments stored. At 2 hours, you need 24 etc., I'd set the value to about 12.2 segments/hour of backup.
--
Jay
Sent from my iPad
> On Dec 5, 2016, at 1:57 PM, Tharmarajah, Sam <sambavan.tharmarajah@usask.ca> wrote:
>
> Hi
>
> I am a newbie to postgres. We are on postgres 9.1 running on RedHat 6. When we do the postgres backups we are noticing the following errors
>
> pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment 000000010000020B00000040 has already been removed
>
> But when I look into pg_xlog, I notice
>
> 000000010000020B00000040.00000118.backup
>
>
> May I know how to resolve this error? Are the backups done with these errors, recoverable?
>
>
> Thanks in advance for your support.
>
> Thanx,
> Sam
>
>
>
>
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment"
2016-12-05 18:57 Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment" Tharmarajah, Sam <sambavan.tharmarajah@usask.ca>
2016-12-05 19:52 ` Re: Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment" John Scalia <jayknowsunix@gmail.com>
@ 2016-12-05 20:27 ` Tharmarajah, Sam <sambavan.tharmarajah@usask.ca>
2016-12-05 20:41 ` Re: Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment" Keith <keith@keithf4.com>
0 siblings, 1 reply; 5+ messages in thread
From: Tharmarajah, Sam @ 2016-12-05 20:27 UTC (permalink / raw)
To: John Scalia <jayknowsunix@gmail.com>; +Cc: pgsql-admin
Hi John,
Thank you for the feed back. Presently it is taking around 1hr to finish the backups. We have the wal_keep_segments set to 50. Is there way to adjust the segment size to be more than 16MB? At times I notice in the pg_xlog, there are 10-15 segments on an invocation point.
Thanx,
Sam
From: John Scalia [mailto:jayknowsunix@gmail.com]
Sent: Monday, December 05, 2016 1:52 PM
To: Tharmarajah, Sam <sambavan.tharmarajah@usask.ca>
Cc: pgsql-admin@postgresql.org
Subject: Re: [ADMIN] Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment"
More than likely your keep_wal_segments setting is too low. How long is the pg_basebackup taking? Remember that segments are generated about every 5 minutes, so if the backup takes an hour, you need at least 12 segments stored. At 2 hours, you need 24 etc., I'd set the value to about 12.2 segments/hour of backup.
--
Jay
Sent from my iPad
On Dec 5, 2016, at 1:57 PM, Tharmarajah, Sam <sambavan.tharmarajah@usask.ca<mailto:sambavan.tharmarajah@usask.ca>> wrote:
Hi
I am a newbie to postgres. We are on postgres 9.1 running on RedHat 6. When we do the postgres backups we are noticing the following errors
pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment 000000010000020B00000040 has already been removed
But when I look into pg_xlog, I notice
000000010000020B00000040.00000118.backup
May I know how to resolve this error? Are the backups done with these errors, recoverable?
Thanks in advance for your support.
Thanx,
Sam
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment"
2016-12-05 18:57 Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment" Tharmarajah, Sam <sambavan.tharmarajah@usask.ca>
2016-12-05 19:52 ` Re: Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment" John Scalia <jayknowsunix@gmail.com>
2016-12-05 20:27 ` Re: Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment" Tharmarajah, Sam <sambavan.tharmarajah@usask.ca>
@ 2016-12-05 20:41 ` Keith <keith@keithf4.com>
2016-12-05 20:51 ` Re: Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment" John Scalia <jayknowsunix@gmail.com>
0 siblings, 1 reply; 5+ messages in thread
From: Keith @ 2016-12-05 20:41 UTC (permalink / raw)
To: Tharmarajah, Sam <sambavan.tharmarajah@usask.ca>; +Cc: John Scalia <jayknowsunix@gmail.com>; pgsql-admin
On Mon, Dec 5, 2016 at 3:27 PM, Tharmarajah, Sam <
sambavan.tharmarajah@usask.ca> wrote:
> Hi John,
>
>
>
> Thank you for the feed back. Presently it is taking around 1hr to finish
> the backups. We have the wal_keep_segments set to 50. Is there way to
> adjust the segment size to be more than 16MB? At times I notice in the
> pg_xlog, there are 10-15 segments on an invocation point.
>
>
>
>
>
> Thanx,
>
> Sam
>
>
>
> *From:* John Scalia [mailto:jayknowsunix@gmail.com]
> *Sent:* Monday, December 05, 2016 1:52 PM
> *To:* Tharmarajah, Sam <sambavan.tharmarajah@usask.ca>
> *Cc:* pgsql-admin@postgresql.org
> *Subject:* Re: [ADMIN] Req. for some help with resolving the error
> "pg_basebackup: could not get WAL end position from server: FATAL:
> requested WAL segment"
>
>
>
> More than likely your keep_wal_segments setting is too low. How long is
> the pg_basebackup taking? Remember that segments are generated about every
> 5 minutes, so if the backup takes an hour, you need at least 12 segments
> stored. At 2 hours, you need 24 etc., I'd set the value to about 12.2
> segments/hour of backup.
>
> --
>
> Jay
>
>
>
> Sent from my iPad
>
>
> On Dec 5, 2016, at 1:57 PM, Tharmarajah, Sam <
> sambavan.tharmarajah@usask.ca> wrote:
>
> Hi
>
>
>
> I am a newbie to postgres. We are on postgres 9.1 running on RedHat 6.
> When we do the postgres backups we are noticing the following errors
>
>
>
> pg_basebackup: could not get WAL end position from server: FATAL:
> requested WAL segment 000000010000020B00000040 has already been removed
>
>
>
> But when I look into pg_xlog, I notice
>
>
>
> 000000010000020B00000040.00000118.backup
>
>
>
>
>
> May I know how to resolve this error? Are the backups done with these
> errors, recoverable?
>
>
>
>
>
> Thanks in advance for your support.
>
>
>
> Thanx,
>
> Sam
>
>
>
>
>
>
>
>
>
>
If you're using pgbasebackup just for creating consistent backups, use the
--xlog-method=strean (-Xs) to ensure you get all the WAL files necessary
for a consistent backup. This avoids having to set wal_keep_segments high
enough to account for the time length of your entire backup. Note you will
need an extra WAL stream for this setting, so ensure max_wal_senders is set
high enough on the master.
Now if you're going to be using this backup to create a slave, you must get
that slave up and running before the master recycles all the WAL files
necessary for replication to be able to catch up from when the backup
stopped. Otherwise you will see that error you shared in your original
post. That's when you may need to adjust wal_keep_segments. Either that or
you should set up an archive_command on the master to keep a secondary copy
of all WAL files.
Keith
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment"
2016-12-05 18:57 Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment" Tharmarajah, Sam <sambavan.tharmarajah@usask.ca>
2016-12-05 19:52 ` Re: Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment" John Scalia <jayknowsunix@gmail.com>
2016-12-05 20:27 ` Re: Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment" Tharmarajah, Sam <sambavan.tharmarajah@usask.ca>
2016-12-05 20:41 ` Re: Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment" Keith <keith@keithf4.com>
@ 2016-12-05 20:51 ` John Scalia <jayknowsunix@gmail.com>
0 siblings, 0 replies; 5+ messages in thread
From: John Scalia @ 2016-12-05 20:51 UTC (permalink / raw)
To: Keith <keith@keithf4.com>; +Cc: Tharmarajah, Sam <sambavan.tharmarajah@usask.ca>; pgsql-admin
The only way to increase the segment size is to adjust the setting in the source code and recompile the server. There are no runtime adjustments available to do this.
Sent from my iPad
> On Dec 5, 2016, at 3:41 PM, Keith <keith@keithf4.com> wrote:
>
>
>
>> On Mon, Dec 5, 2016 at 3:27 PM, Tharmarajah, Sam <sambavan.tharmarajah@usask.ca> wrote:
>> Hi John,
>>
>>
>>
>> Thank you for the feed back. Presently it is taking around 1hr to finish the backups. We have the wal_keep_segments set to 50. Is there way to adjust the segment size to be more than 16MB? At times I notice in the pg_xlog, there are 10-15 segments on an invocation point.
>>
>>
>>
>>
>>
>> Thanx,
>>
>> Sam
>>
>>
>>
>> From: John Scalia [mailto:jayknowsunix@gmail.com]
>> Sent: Monday, December 05, 2016 1:52 PM
>> To: Tharmarajah, Sam <sambavan.tharmarajah@usask.ca>
>> Cc: pgsql-admin@postgresql.org
>> Subject: Re: [ADMIN] Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment"
>>
>>
>>
>> More than likely your keep_wal_segments setting is too low. How long is the pg_basebackup taking? Remember that segments are generated about every 5 minutes, so if the backup takes an hour, you need at least 12 segments stored. At 2 hours, you need 24 etc., I'd set the value to about 12.2 segments/hour of backup.
>>
>> --
>>
>> Jay
>>
>>
>>
>> Sent from my iPad
>>
>>
>> On Dec 5, 2016, at 1:57 PM, Tharmarajah, Sam <sambavan.tharmarajah@usask.ca> wrote:
>>
>> Hi
>>
>>
>>
>> I am a newbie to postgres. We are on postgres 9.1 running on RedHat 6. When we do the postgres backups we are noticing the following errors
>>
>>
>>
>> pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment 000000010000020B00000040 has already been removed
>>
>>
>>
>> But when I look into pg_xlog, I notice
>>
>>
>>
>> 000000010000020B00000040.00000118.backup
>>
>>
>>
>>
>>
>> May I know how to resolve this error? Are the backups done with these errors, recoverable?
>>
>>
>>
>>
>>
>> Thanks in advance for your support.
>>
>>
>>
>> Thanx,
>>
>> Sam
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
> If you're using pgbasebackup just for creating consistent backups, use the --xlog-method=strean (-Xs) to ensure you get all the WAL files necessary for a consistent backup. This avoids having to set wal_keep_segments high enough to account for the time length of your entire backup. Note you will need an extra WAL stream for this setting, so ensure max_wal_senders is set high enough on the master.
>
> Now if you're going to be using this backup to create a slave, you must get that slave up and running before the master recycles all the WAL files necessary for replication to be able to catch up from when the backup stopped. Otherwise you will see that error you shared in your original post. That's when you may need to adjust wal_keep_segments. Either that or you should set up an archive_command on the master to keep a secondary copy of all WAL files.
>
> Keith
^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2016-12-05 20:51 UTC | newest]
Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2016-12-05 18:57 Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment" Tharmarajah, Sam <sambavan.tharmarajah@usask.ca>
2016-12-05 19:52 ` John Scalia <jayknowsunix@gmail.com>
2016-12-05 20:27 ` Tharmarajah, Sam <sambavan.tharmarajah@usask.ca>
2016-12-05 20:41 ` Re: Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment" Keith <keith@keithf4.com>
2016-12-05 20:51 ` Re: Req. for some help with resolving the error "pg_basebackup: could not get WAL end position from server: FATAL: requested WAL segment" John Scalia <jayknowsunix@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox