pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedMissing timeline history file after execution of pg_upgrade
8+ messages / 4 participants
[nested] [flat]
* Missing timeline history file after execution of pg_upgrade
@ 2014-11-10 13:48 christian.echerer@manroland-web.com
0 siblings, 1 reply; 8+ messages in thread
From: christian.echerer@manroland-web.com @ 2014-11-10 13:48 UTC (permalink / raw)
To: pgsql-admin
Hi,
after execution of pg_upgrade is the timeline history file (i.e. 00000002.history) in the new database cluster missing. Therefore, it is no longer possible to use this database cluster as the master for a streaming replication client.
Test description:
-----------------
- Running postgres database (Version 9.3.4)
- Setting Up WAL Archiving for this database named "Database A"
- Making a base backup of "Database A"
- Crash of "Database A"
- Do a recovering using a "Continuous archive backup" of "Database A"
(new timeline is created)
- Listing of data/pg_xlog (Database A) after recovering
-- 16777216 Oct 22 13:36 000000010000000000000006
-- 16777216 Oct 22 13:36 000000010000000000000007
-- 16777216 Oct 22 13:36 000000010000000000000008
-- 41 Oct 22 13:36 00000002.history
-- 16777216 Oct 22 13:36 000000020000000000000008
-- 16777216 Oct 22 13:38 000000020000000000000009
-- 16777216 Oct 22 13:54 00000002000000000000000A
-- 16777216 Oct 22 13:54 00000002000000000000000B
-- 16777216 Oct 22 13:54 00000002000000000000000C
-- 4096 Oct 22 13:54 archive_status/
- Install a new Postgres database (Version 9.3.4) named "Database B"
- Data transfer using pg_upgrade (Database A -> Database B)
(pg_upgrade is not necessary, but it would be a standard procedure, that should work also with the same Postgres version)
- Listing of data/pg_xlog (Database B)
-- 16777216 Oct 29 06:00 00000002000000000000000D
-- 16777216 Nov 5 06:00 00000002000000000000000E
-- 16777216 Nov 10 14:08 00000002000000000000000F
-- 4096 Oct 22 13:54 archive_status/
- Setup a streaming replication client named "Database C" on "Database B"
- Error occurs:
... : could not receive timeline history file from the primary server: ERROR: could not open file "pg_xlog/00000002.history": No such file or directory
I have three questions:
1) Why pg_upgrade changed the timeline informations?
2) How do I get replication (Database B -> Database C) working?
3) Is this behavior intended. Is there a workaround for me?
Please help me to troubleshoot this problem. I will feedback any required information.
Thanks in advance
Christian
________________________________
manroland web systems GmbH -- Managing Director: Joern Gosse
Registered Office: Augsburg -- Trade Register: AG Augsburg -- HRB-No.: 26816 -- VAT: DE281389840
Confidentiality note:
This eMail and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the intended recipient, you are hereby notified that any use or dissemination of this communication is strictly prohibited. If you have received this eMail in error, then please delete this eMail.
! Please consider your environmental responsibility before printing this eMail !
________________________________
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Missing timeline history file after execution of pg_upgrade
@ 2014-12-06 14:07 Bruce Momjian <bruce@momjian.us>
parent: christian.echerer@manroland-web.com
0 siblings, 1 reply; 8+ messages in thread
From: Bruce Momjian @ 2014-12-06 14:07 UTC (permalink / raw)
To: christian.echerer@manroland-web.com; +Cc: pgsql-admin
On Mon, Nov 10, 2014 at 01:48:40PM +0000, christian.echerer@manroland-web.com wrote:
> I have three questions:
> 1) Why pg_upgrade changed the timeline informations?
> 2) How do I get replication (Database B -> Database C) working?
> 3) Is this behavior intended. Is there a workaround for me?
You need to create new standbys after the upgrade --- pg_upgrade does
not retain the WAL files from the previous cluster, as the format might
have changed.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Missing timeline history file after execution of pg_upgrade
@ 2015-02-02 10:07 christian.echerer@manroland-web.com
parent: Bruce Momjian <bruce@momjian.us>
0 siblings, 2 replies; 8+ messages in thread
From: christian.echerer@manroland-web.com @ 2015-02-02 10:07 UTC (permalink / raw)
To: bruce@momjian.us; +Cc: pgsql-admin
On 06.12.2014, bruce@momjian.us wrote:
> You need to create new standbys after the upgrade --- pg_upgrade does not retain the WAL files from the previous cluster, as the format might have changed.
I created a new standby (pg_start_backup(), rsync, ...) after the execution of pg_upgrade on the master. On the master is the timeline history missing and i get the following error message when I try to set up a streaming client:
... : could not receive timeline history file from the primary server: ERROR: could not open file "pg_xlog/00000002.history": No such file or directory
Short test description (Postgres-Version 9.3.4)
--------------------------------------
- Do a recovering on the Master (new timeline is created)
- Execute pg_upgrade on the Master (timeline informations are missing)
- Try to setup a new streaming client on the master
- Error occurs: ... : could not receive timeline history file from the primary server: ERROR: could not open file "pg_xlog/00000002.history": No such file or directory
A longer description can be found in the mail from 11.10.2014.
How do I get streaming replication working again?
My current workaround is to restore the master with the help of pg_restore, after updating the database server version. It takes quite a long time, however, and is therefore a problem.
Thanks in advance
Christian
________________________________
manroland web systems GmbH -- Managing Director: Joern Gosse
Registered Office: Augsburg -- Trade Register: AG Augsburg -- HRB-No.: 26816 -- VAT: DE281389840
Confidentiality note:
This eMail and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the intended recipient, you are hereby notified that any use or dissemination of this communication is strictly prohibited. If you have received this eMail in error, then please delete this eMail.
! Please consider your environmental responsibility before printing this eMail !
________________________________
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Missing timeline history file after execution of pg_upgrade
@ 2015-02-03 09:36 dezso <avandras@gmail.com>
parent: christian.echerer@manroland-web.com
1 sibling, 0 replies; 8+ messages in thread
From: dezso @ 2015-02-03 09:36 UTC (permalink / raw)
To: pgsql-admin
Just to confirm that this problem exists: we upgraded recently from 9.3 to
9.4 using pg_upgrade, having the slave rebuilt as prescribed. Directly
after starting up the slave, we started getting the above mentioned error.
Replication to this slave works otherwise.
Is there a way to get the history file back?
--
View this message in context: http://postgresql.nabble.com/Missing-timeline-history-file-after-execution-of-pg-upgrade-tp5826326p5...
Sent from the PostgreSQL - admin mailing list archive at Nabble.com.
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Missing timeline history file after execution of pg_upgrade
@ 2015-02-03 21:41 Alexey Klyukin <alexk@hintbits.com>
parent: christian.echerer@manroland-web.com
1 sibling, 1 reply; 8+ messages in thread
From: Alexey Klyukin @ 2015-02-03 21:41 UTC (permalink / raw)
To: christian.echerer@manroland-web.com; avandras@gmail.com; +Cc: pgsql-admin
Hi,
On Mon, Feb 2, 2015 at 11:07 AM, <christian.echerer@manroland-web.com> wrote:
>
> Short test description (Postgres-Version 9.3.4)
> --------------------------------------
> - Do a recovering on the Master (new timeline is created)
> - Execute pg_upgrade on the Master (timeline informations are missing)
> - Try to setup a new streaming client on the master
> - Error occurs: ... : could not receive timeline history file from the primary server: ERROR: could not open file "pg_xlog/00000002.history": No such file or directory
>
> A longer description can be found in the mail from 11.10.2014.
>
> How do I get streaming replication working again?
>
> My current workaround is to restore the master with the help of pg_restore, after updating the database server version. It takes quite a long time, however, and is therefore a problem.
The path to reproduce the problem is quite simple:
- setup the master and the replica setup, and promote the replica,
creating a new timeline (timeline 2 by default)
- pg_upgrade the promoted replica. This would create a new cluster
with the same timeline, but without the history file.
- create the streaming-only replica. The streaming replication will
never start, since the timeline file is not there. The error messages
are:
ERROR: could not open file "pg_xlog/00000002.history": No such file
or directory
FATAL: could not receive timeline history file from the primary
server: ERROR: could not open file "pg_xlog/00000002.history": No
such file or directory
Note that this problem does not occur for the streaming-only replica
of the newly created cluster (with a timeline 1), even when there is
no timeline history file in the original datadir.
--
Regards,
Alexey Klyukin
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Missing timeline history file after execution of pg_upgrade
@ 2015-02-04 08:23 Alexey Klyukin <alexk@hintbits.com>
parent: Alexey Klyukin <alexk@hintbits.com>
0 siblings, 1 reply; 8+ messages in thread
From: Alexey Klyukin @ 2015-02-04 08:23 UTC (permalink / raw)
To: christian.echerer@manroland-web.com, András Váczi <avandras@gmail.com>; +Cc: Bruce Momjian <bruce@momjian.us>; pgsql-admin
On Tue, Feb 3, 2015 at 10:41 PM, Alexey Klyukin <alexk@hintbits.com> wrote:
> The path to reproduce the problem is quite simple:
>
> - setup the master and the replica setup, and promote the replica,
> creating a new timeline (timeline 2 by default)
> - pg_upgrade the promoted replica. This would create a new cluster
> with the same timeline, but without the history file.
> - create the streaming-only replica. The streaming replication will
> never start, since the timeline file is not there. The error messages
> are:
>
> ERROR: could not open file "pg_xlog/00000002.history": No such file
> or directory
> FATAL: could not receive timeline history file from the primary
> server: ERROR: could not open file "pg_xlog/00000002.history": No
> such file or directory
>
> Note that this problem does not occur for the streaming-only replica
> of the newly created cluster (with a timeline 1), even when there is
> no timeline history file in the original datadir.
In our case I solved the problem by manually creating a 00000002.history file
on master. It consists of the previous timeline (obviously 1), the location when
the new timeline has been created (lost) and the text reason, which can be
arbitrary. As for the location, I don't think it matters much, since
we don't have
branched timelines and not going to switch to timeline 1 again, I've
put the value
of the 'Prior checkpoint location' from pg_controldata on the replica,
but I think
I could just put all 0 with the same effect.
Nevertheless, the question is should pg_upgrade move the timeline history file
from the old server to the new one, and, to be more specific, are
there any cases when
moving such file is not recommended?
Kind regards,
--
Alexey Klyukin
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: Missing timeline history file after execution of pg_upgrade
@ 2015-05-15 02:35 Bruce Momjian <bruce@momjian.us>
parent: Alexey Klyukin <alexk@hintbits.com>
0 siblings, 1 reply; 8+ messages in thread
From: Bruce Momjian @ 2015-05-15 02:35 UTC (permalink / raw)
To: Alexey Klyukin <alexk@hintbits.com>; +Cc: christian.echerer@manroland-web.com, András Váczi <avandras@gmail.com>; pgsql-admin
On Wed, Feb 4, 2015 at 09:23:56AM +0100, Alexey Klyukin wrote:
> On Tue, Feb 3, 2015 at 10:41 PM, Alexey Klyukin <alexk@hintbits.com> wrote:
> > The path to reproduce the problem is quite simple:
> >
> > - setup the master and the replica setup, and promote the replica,
> > creating a new timeline (timeline 2 by default)
> > - pg_upgrade the promoted replica. This would create a new cluster
> > with the same timeline, but without the history file.
> > - create the streaming-only replica. The streaming replication will
> > never start, since the timeline file is not there. The error messages
> > are:
> >
> > ERROR: could not open file "pg_xlog/00000002.history": No such file
> > or directory
> > FATAL: could not receive timeline history file from the primary
> > server: ERROR: could not open file "pg_xlog/00000002.history": No
> > such file or directory
> >
> > Note that this problem does not occur for the streaming-only replica
> > of the newly created cluster (with a timeline 1), even when there is
> > no timeline history file in the original datadir.
>
>
> In our case I solved the problem by manually creating a 00000002.history file
> on master. It consists of the previous timeline (obviously 1), the location when
> the new timeline has been created (lost) and the text reason, which can be
> arbitrary. As for the location, I don't think it matters much, since
> we don't have
> branched timelines and not going to switch to timeline 1 again, I've
> put the value
> of the 'Prior checkpoint location' from pg_controldata on the replica,
> but I think
> I could just put all 0 with the same effect.
>
> Nevertheless, the question is should pg_upgrade move the timeline history file
> from the old server to the new one, and, to be more specific, are
> there any cases when
> moving such file is not recommended?
Sorry I am just getting to this --- you are right it is a bug. I was
unaware that pg_upgrade passed the timeline from the old cluster, but it
does so when setting the WAL starting address with pg_resetxlog -l,
which includes the timeline as the first eight hex digits.
I think there are two options:
1) force pg_resetxlog -l to pass timeline 1
2) create a WAL history file to match the old cluster's timeline (!= 1)
I don't think we want to be doing #2, so I have developed the attached
patch for #1, which I should backpatch to 9.4. Do we store the timeline
in any user tables that might be transfered? Anywhere else?
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
Attachments:
timeline.diff (text/x-diff, 948B) download
diff --git a/src/bin/pg_upgrade/pg_upgrade.c b/src/bin/pg_upgrade/pg_upgrade.c
new file mode 100644
index fbccc2e..cc81fa0
*** a/src/bin/pg_upgrade/pg_upgrade.c
--- b/src/bin/pg_upgrade/pg_upgrade.c
*************** copy_clog_xlog_xid(void)
*** 473,480 ****
/* now reset the wal archives in the new cluster */
prep_status("Resetting WAL archives");
exec_prog(UTILITY_LOG_FILE, NULL, true,
! "\"%s/pg_resetxlog\" -l %s \"%s\"", new_cluster.bindir,
! old_cluster.controldata.nextxlogfile,
new_cluster.pgdata);
check_ok();
}
--- 473,481 ----
/* now reset the wal archives in the new cluster */
prep_status("Resetting WAL archives");
exec_prog(UTILITY_LOG_FILE, NULL, true,
! /* use timeline 1 to match controldata and no WAL history file */
! "\"%s/pg_resetxlog\" -l 00000001%s \"%s\"", new_cluster.bindir,
! old_cluster.controldata.nextxlogfile + 8,
new_cluster.pgdata);
check_ok();
}
* Re: Missing timeline history file after execution of pg_upgrade
@ 2015-05-16 04:40 Bruce Momjian <bruce@momjian.us>
parent: Bruce Momjian <bruce@momjian.us>
0 siblings, 0 replies; 8+ messages in thread
From: Bruce Momjian @ 2015-05-16 04:40 UTC (permalink / raw)
To: Alexey Klyukin <alexk@hintbits.com>; +Cc: christian.echerer@manroland-web.com, András Váczi <avandras@gmail.com>; pgsql-admin
On Thu, May 14, 2015 at 10:35:29PM -0400, Bruce Momjian wrote:
> >
> > Nevertheless, the question is should pg_upgrade move the timeline history file
> > from the old server to the new one, and, to be more specific, are
> > there any cases when
> > moving such file is not recommended?
>
> Sorry I am just getting to this --- you are right it is a bug. I was
> unaware that pg_upgrade passed the timeline from the old cluster, but it
> does so when setting the WAL starting address with pg_resetxlog -l,
> which includes the timeline as the first eight hex digits.
>
> I think there are two options:
>
> 1) force pg_resetxlog -l to pass timeline 1
> 2) create a WAL history file to match the old cluster's timeline (!= 1)
>
> I don't think we want to be doing #2, so I have developed the attached
> patch for #1, which I should backpatch to 9.4. Do we store the timeline
> in any user tables that might be transfered? Anywhere else?
Patch applied and backpatched through 9.0.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
^ permalink raw reply [nested|flat] 8+ messages in thread
end of thread, other threads:[~2015-05-16 04:40 UTC | newest]
Thread overview: 8+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2014-11-10 13:48 Missing timeline history file after execution of pg_upgrade christian.echerer@manroland-web.com
2014-12-06 14:07 ` Bruce Momjian <bruce@momjian.us>
2015-02-02 10:07 ` christian.echerer@manroland-web.com
2015-02-03 09:36 ` dezso <avandras@gmail.com>
2015-02-03 21:41 ` Alexey Klyukin <alexk@hintbits.com>
2015-02-04 08:23 ` Alexey Klyukin <alexk@hintbits.com>
2015-05-15 02:35 ` Bruce Momjian <bruce@momjian.us>
2015-05-16 04:40 ` Bruce Momjian <bruce@momjian.us>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox