From: PG Bug reporting form <noreply@postgresql.org>
To: pgsql-bugs@lists.postgresql.org
Cc: mr.kaleek@gmail.com
Subject: BUG #16208: background worker "logical replication worker" was terminated by signal 11: Segmentation
Date: Thu, 16 Jan 2020 12:26:08 +0000
Message-ID: <16208-784ecaeb269906e4@postgresql.org> (raw)
The following bug has been logged on the website:
Bug reference: 16208
Logged by: Firdaus Firdaus
Email address: mr.kaleek@gmail.com
PostgreSQL version: 12.1
Operating system: Ubuntu 16.04.6 LTS
Description:
I got error when replicate with native logical method from PostgreSQL 11.6
to PostgreSQL 12.1, I will connecting table partition to table partition
from Version 11.6 to Version 12.1, but after all tables already replicated
around more than 2 hours, server can't access and recovery mode.
Actually we are facing problem when replicating partition table from 11.6 to
12.1, but if there is no partition table the replication works so well.
As FYI, our server spec is 16 core and 64GB. And we are using this ubuntu
LSB Version:
core-9.20160110ubuntu0.2-amd64:core-9.20160110ubuntu0.2-noarch:security-9.20160110ubuntu0.2-amd64:security-9.20160110ubuntu0.2-noarch
Distributor ID: Ubuntu
Description: Ubuntu 16.04.6 LTS
Release: 16.04
Codename: xenial
Can you help us to find out the solution and configuration?
please find below :
sudo gdb -p 15803
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html;
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/;.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/;.
For help, type "help".
Type "apropos word" to search for commands related to "word".
Attaching to process 15803
Reading symbols from /usr/lib/postgresql/12/bin/postgres...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libpthread.so.0...Reading symbols
from
/usr/lib/debug/.build-id/b1/7c21299099640a6d863e423d99265824e7bb16.debug...done.
done.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Reading symbols from /usr/lib/x86_64-linux-gnu/libxml2.so.2...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libpam.so.0...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libssl.so.1.0.0...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2...(no
debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/librt.so.1...Reading symbols from
/usr/lib/debug//lib/x86_64-linux-gnu/librt-2.23.so...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libdl.so.2...Reading symbols from
/usr/lib/debug//lib/x86_64-linux-gnu/libdl-2.23.so...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libm.so.6...Reading symbols from
/usr/lib/debug//lib/x86_64-linux-gnu/libm-2.23.so...done.
done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libicui18n.so.55...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libicuuc.so.55...(no
debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libsystemd.so.0...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libc.so.6...Reading symbols from
/usr/lib/debug//lib/x86_64-linux-gnu/libc-2.23.so...done.
done.
Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from
/usr/lib/debug//lib/x86_64-linux-gnu/ld-2.23.so...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libz.so.1...(no debugging symbols
found)...done.
Reading symbols from /lib/x86_64-linux-gnu/liblzma.so.5...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libaudit.so.1...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libkrb5.so.3...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libk5crypto.so.3...(no
debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libcom_err.so.2...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libkrb5support.so.0...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2...(no
debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libresolv.so.2...Reading symbols
from /usr/lib/debug//lib/x86_64-linux-gnu/libresolv-2.23.so...done.
done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libsasl2.so.2...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libgssapi.so.3...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libgnutls.so.30...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libstdc++.so.6...(no
debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libgcc_s.so.1...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libicudata.so.55...(no
debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libselinux.so.1...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libgcrypt.so.20...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libkeyutils.so.1...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libheimntlm.so.0...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libkrb5.so.26...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libasn1.so.8...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libhcrypto.so.4...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libroken.so.18...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libp11-kit.so.0...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libidn.so.11...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libtasn1.so.6...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libnettle.so.6...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libhogweed.so.4...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libgmp.so.10...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libpcre.so.3...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libgpg-error.so.0...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libwind.so.0...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libheimbase.so.1...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libhx509.so.5...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libsqlite3.so.0...(no
debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libcrypt.so.1...Reading symbols
from /usr/lib/debug//lib/x86_64-linux-gnu/libcrypt-2.23.so...done.
done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libffi.so.6...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/postgresql/12/lib/pg_stat_statements.so...(no
debugging symbols found)...done.
Reading symbols from /usr/lib/postgresql/12/lib/pg_partman_bgw.so...(no
debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libnss_files.so.2...Reading
symbols from
/usr/lib/debug//lib/x86_64-linux-gnu/libnss_files-2.23.so...done.
done.
0x00007f5ce4bf85b3 in __select_nocancel () at
../sysdeps/unix/syscall-template.S:84
84 ../sysdeps/unix/syscall-template.S: No such file or directory.
(gdb) cont
Continuing.
Program received signal SIGUSR1, User defined signal 1.
0x00007f5ce4b30730 in __sigprocmask (how=2, set=0x56135acfba40 <UnBlockSig>,
oset=0x0)
at ../sysdeps/unix/sysv/linux/x86_64/sigprocmask.c:36
36 ../sysdeps/unix/sysv/linux/x86_64/sigprocmask.c: No such file or
directory.
(gdb) detach
Detaching from program: /usr/lib/postgresql/12/bin/postgres, process 15803
(gdb) b errfinish
Breakpoint 1 at 0x56135a814a94
$ for f in `pgrep postgres`; do ls -l /usr/lib/postgresql/12/bin/postgres;
done
-rwxr-xr-x 1 root root 7978032 Nov 12 19:34
/usr/lib/postgresql/12/bin/postgres
-rwxr-xr-x 1 root root 7978032 Nov 12 19:34
/usr/lib/postgresql/12/bin/postgres
-rwxr-xr-x 1 root root 7978032 Nov 12 19:34
/usr/lib/postgresql/12/bin/postgres
$./pg_ctl -D /data/postgres/data -l logfile -c start
-su: ./pg_ctl: No such file or directory
$ pg_ctl -D /data/postgres/data -l logfile -c start
pg_ctl: command not found
on datadir I did not see core.PID, let me know where the dir.
$ ls -lrth
total 144K
drwx------ 2 postgres postgres 4.0K Jan 2 16:34 pg_twophase
drwx------ 2 postgres postgres 4.0K Jan 2 16:34 pg_tblspc
drwx------ 2 postgres postgres 4.0K Jan 2 16:34 pg_snapshots
drwx------ 2 postgres postgres 4.0K Jan 2 16:34 pg_serial
drwx------ 4 postgres postgres 4.0K Jan 2 16:34 pg_multixact
drwx------ 2 postgres postgres 4.0K Jan 2 16:34 pg_dynshmem
drwx------ 2 postgres postgres 4.0K Jan 2 16:34 pg_commit_ts
-rw------- 1 postgres postgres 3 Jan 2 16:34 PG_VERSION
-rw------- 1 postgres postgres 88 Jan 2 16:34 postgresql.auto.conf
drwx------ 2 postgres postgres 4.0K Jan 2 16:34 pg_xact
drwx------ 2 postgres postgres 4.0K Jan 2 16:34 pg_subtrans
drwx------ 6 postgres postgres 4.0K Jan 2 16:44 base
drwx------ 5 postgres postgres 4.0K Jan 3 18:29 pg_replslot
-rw------- 1 postgres postgres 127 Jan 15 12:38 postmaster.opts
drwx------ 2 postgres postgres 4.0K Jan 16 01:00 pg_log
-rw------- 1 postgres postgres 47 Jan 16 01:00 current_logfiles
drwx------ 3 postgres postgres 52K Jan 16 11:00 pg_wal
drwx------ 4 postgres postgres 4.0K Jan 16 11:40 pg_logical
-rw------- 1 postgres postgres 93 Jan 16 11:40 postmaster.pid
drwx------ 2 postgres postgres 4.0K Jan 16 11:40 pg_stat_tmp
drwx------ 2 postgres postgres 4.0K Jan 16 11:40 pg_notify
drwx------ 2 postgres postgres 4.0K Jan 16 11:40 global
drwx------ 2 postgres postgres 4.0K Jan 16 11:40 pg_stat
$ cd postmaster.pid
-su: cd: postmaster.pid: Not a directory
$ more postmaster.pid
15803
/data/postgres/data
1579063112
5432
/var/run/postgresql
*
5432001 169312256
ready
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-bugs@postgresql.org
Cc: noreply@postgresql.org, mr.kaleek@gmail.com
Subject: Re: BUG #16208: background worker "logical replication worker" was terminated by signal 11: Segmentation
In-Reply-To: <16208-784ecaeb269906e4@postgresql.org>
References: <16208-784ecaeb269906e4@postgresql.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox