From: Laurenz Albe <laurenz.albe@cybertec.at>
To: Douglas Reed <douglas@fsbtech.com>
To: pgsql-admin@postgresql.org
Subject: Re: Problems rebuilding slave using pg_basebackup
Date: Wed, 08 Nov 2017 13:28:32 +0100
Message-ID: <1510144112.2752.24.camel@cybertec.at> (raw)
In-Reply-To: <CAN7N4m3jAfnWV+1jq7-6S9wowEHU6Kdoomh3Rfe_RdSdt0sDfg@mail.gmail.com>
References: <CAN7N4m3jAfnWV+1jq7-6S9wowEHU6Kdoomh3Rfe_RdSdt0sDfg@mail.gmail.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-admin>
Douglas Reed wrote:
> We are having a number of problems when we attempt to rebuild our slave from its master
>
> We have made about three attempts without success (using a proven set of notes)
>
> It's been rebuilt several times over the last few months although the time between
> pg_basebackup being keyed and it actually copying data can be up to six minutes.
> And after completion the time taken from database startup to psql availability
> can also be several minutes while it processes any remaining logs.
>
> Both machines are virtuals and are based with a leading cloud provider
>
> OS Linux Centos6 (6.8 Final)
>
> pg version 9.5.4
[...]
> Message from pg_basebackup
>
> [postgres@xxxxxxxxxx]$ pg_basebackup -h -IP_HIDDEN- -D /var/lib/pgsql/9.5/data -P -U postgres --xlog-method=stream
> pg_basebackup: could not receive data from WAL stream: server closed the connection unexpectedly
> This probably means the server terminated abnormally
> before or while processing the request.
> 269061959/269164935 kB (99%), 1/1 tablespace
> pg_basebackup: child process exited with error 1
>
>
> Relevant error messages from master's log
>
> Nov 7 11:52:32 o8-data1 postgres[28558]: [6-1] user=[unknown],db=[unknown],app=[unknown]client=-IP_HIDDEN- LOG: connection received: host=-IP_HIDDEN- port=41498
> Nov 7 11:52:32 o8-data1 postgres[28558]: [7-1] user=postgres,db=[unknown],app=[unknown]client=-IP_HIDDEN- LOG: replication connection authorized: user=postgres
> Nov 7 13:51:44 o8-data1 postgres[28558]: [8-1] user=postgres,db=[unknown],app=pg_basebackupclient=-IP_HIDDEN- LOG: could not send data to client: Broken pipe
> Nov 7 13:51:44 o8-data1 postgres[28558]: [9-1] user=postgres,db=[unknown],app=pg_basebackupclient=-IP_HIDDEN- ERROR: base backup could not send data, aborting backup
> Nov 7 13:51:44 o8-data1 postgres[28558]: [10-1] user=postgres,db=[unknown],app=pg_basebackupclient=-IP_HIDDEN- FATAL: connection to client lost
> Nov 7 13:51:44 o8-data1 postgres[28558]: [11-1] user=postgres,db=[unknown],app=pg_basebackupclient=-IP_HIDDEN- LOG: disconnection: session time: 1:59:11.943 user=postgres database= host=-IP_HIDDEN- port=41498
>
> Nov 7 13:54:48 o8-data1 postgres[35445]: [6-1] user=[unknown],db=[unknown],app=[unknown]client=-IP_HIDDEN- LOG: connection received: host=-IP_HIDDEN- port=44040
> Nov 7 13:54:48 o8-data1 postgres[35445]: [7-1] user=postgres,db=[unknown],app=[unknown]client=-IP_HIDDEN- LOG: replication connection authorized: user=postgres
> Nov 7 15:09:20 o8-data1 postgres[35445]: [8-1] user=postgres,db=[unknown],app=pg_basebackupclient=-IP_HIDDEN- LOG: could not send data to client: Broken pipe
> Nov 7 15:09:20 o8-data1 postgres[35445]: [9-1] user=postgres,db=[unknown],app=pg_basebackupclient=-IP_HIDDEN- ERROR: base backup could not send data, aborting backup
> Nov 7 15:09:20 o8-data1 postgres[35445]: [10-1] user=postgres,db=[unknown],app=pg_basebackupclient=-IP_HIDDEN- FATAL: connection to client lost
> Nov 7 15:09:20 o8-data1 postgres[35445]: [11-1] user=postgres,db=[unknown],app=pg_basebackupclient=-IP_HIDDEN- LOG: disconnection: session time: 1:14:31.925 user=postgres database= host=-IP_HIDDEN- port=44040
Both client and server complain that the peer has gone away.
That means that the network connection got interrupted.
Either you do not have a reliable TCP connection, or some in-between firewall
terminates the connection.
Yours,
Laurenz Albe
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: laurenz.albe@cybertec.at, douglas@fsbtech.com
Subject: Re: Problems rebuilding slave using pg_basebackup
In-Reply-To: <1510144112.2752.24.camel@cybertec.at>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox