pg.ddx.io  pgsql-bugs@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
To: Burgess, Freddie <Freddie.Burgess@maxar.com>
Cc: pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>
Subject: Re: pg_dump error attempting to upgrade from PostgreSQL 10 to PostgreSQL 12
Date: Sat, 7 Nov 2020 01:10:04 +0100
Message-ID: <20201107011004.50b6de0f@enterprisedb.com> (raw)
In-Reply-To: <DM3P110MB0265AC329BFBD64AB2ADF78BE6EE0@DM3P110MB0265.NAMP110.PROD.OUTLOOK.COM>
References: <DM3P110MB0265AC329BFBD64AB2ADF78BE6EE0@DM3P110MB0265.NAMP110.PROD.OUTLOOK.COM>

On Thu, 5 Nov 2020 21:19:17 +0000
"Burgess, Freddie" <Freddie.Burgess@maxar.com> wrote:

> Simple steps:
> 
> BACKUP: pg_dump -U postgres -d <database> > sherlock.dmp  <- From the
> pg10 instance RESTORE: psql -U postgres -d <database> -1 -f
> sherlock.dmp <- On the pg12 instance
> 
> Postgres Log:
> 
> free(): invalid pointer
> free(): invalid pointer
> 2020-11-05 14:07:33.784 EST [26] LOG:  background worker "parallel
> worker" (PID 150) was terminated by signal 6: Aborted 2020-11-05
> 14:07:33.784 EST [26] LOG:  terminating any other active server
> processes 2020-11-05 14:07:33.784 EST [32] WARNING:  terminating
> connection because of crash of another server process 2020-11-05
> 14:07:33.784 EST [32] DETAIL:  The postmaster has commanded this
> server process to roll back the current transaction and exit, because
> another server process exited abnormally and possibly corrupted
> shared memory. 2020-11-05 14:07:33.784 EST [32] HINT:  In a moment
> you should be able to reconnect to the database and repeat your
> command. 2020-11-05 14:07:33.784 EST [61] WARNING:  terminating
> connection because of crash of another server process 2020-11-05
> 14:07:33.784 EST [61] DETAIL:  The postmaster has commanded this
> server process to roll back the current transaction and exit, because
> another server process exited abnormally and possibly corrupted
> shared memory. 2020-11-05 14:07:33.784 EST [61] HINT:  In a moment
> you should be able to reconnect to the database and repeat your
> command. 2020-11-05 14:07:34.699 EST [26] LOG:  all server processes
> terminated; reinitializing 2020-11-05 14:07:42.266 EST [154] LOG:
> database system was interrupted; last known up at 2020-11-05 14:06:02
> EST 2020-11-05 14:08:05.855 EST [154] LOG:  database system was not
> properly shut down; automatic recovery in progress 2020-11-05
> 14:08:05.859 EST [154] LOG:  redo starts at 7E/93B22C8 2020-11-05
> 14:08:15.931 EST [154] LOG:  invalid record length at 7F/74ECBE30:
> wanted 24, got 0 2020-11-05 14:08:15.931 EST [154] LOG:  redo done at
> 7F/74ECBDF8 2020-11-05 14:08:41.673 EST [26] LOG:  database system is
> ready to accept connections
> 
> PostgreSQL is installed on a docker container, running on a EC2
> instance with 256 GB of memory
> 

It'd be interesting to know what is doing the crashing parallel worker.
Considering it's a background worker, the easiest way is probably
enabling core dumps and inspecting them with gdb. Make sure you have
debug symbols installed and send us the backtrace.

Some basic instructions are in:

https://wiki.postgresql.org/wiki/Getting_a_stack_trace_of_a_running_PostgreSQL_backend_on_Linux/BSD#...

The error message is most likely a random glibc free() error, not sure
where it's coming from or whether it has something to do with docker.

Maybe try preparing a reproducer, i.e. a small database triggering the
issue, which we might use to reproduce the issue on our machines.


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company





view thread (6+ messages)  latest in thread

Message-ID: <20201107011004.50b6de0f@enterprisedb.com>
Permalink:  ../20201107011004.50b6de0f@enterprisedb.com/
Also on:    postgresql.org/message-id/20201107011004.50b6de0f@enterprisedb.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-bugs@postgresql.org
  Cc: tomas.vondra@enterprisedb.com, Freddie.Burgess@maxar.com, pgsql-bugs@lists.postgresql.org
  Subject: Re: pg_dump error attempting to upgrade from PostgreSQL 10 to PostgreSQL 12
  In-Reply-To: <20201107011004.50b6de0f@enterprisedb.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox