From: "Ferrell, Denise SDC" <denise.ferrell@sdc-world.com>
To: Kevin Grittner <kgrittn@gmail.com>
CC: "pgsql-admin@postgresql.org" <pgsql-admin@postgresql.org>
Subject: Re: Data Corruption Errors
Thread-Topic: Data Corruption Errors
Thread-Index: AQHUVl3LwNXdBIgBL02J/AxJCGwLx6UEYSuA///HJE4=
Date: Thu, 27 Sep 2018 14:01:22 +0000
Message-ID: <1538056882208.60158@sdc-world.com>
References: 
 <CAFwQ8rdGB-jUPtrDXpp-gMgS8guWg8SWSkBjkXy0San3rmvj8w@mail.gmail.com>
 <64b2d16e-6c7c-002a-1e16-209a3ae43bef@2ndquadrant.com>
 <1538051973729.67553@sdc-world.com>,<CACjxUsP_qz_jOkNd2-Ybg10uP5jGYwst-5iV+0O9Spye++Ad9g@mail.gmail.com>
In-Reply-To: 
 <CACjxUsP_qz_jOkNd2-Ybg10uP5jGYwst-5iV+0O9Spye++Ad9g@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Precedence: bulk

This seems to have been the issue.  Thank you for the responses.=0A=
________________________________________=0A=
From: Kevin Grittner <kgrittn@gmail.com>=0A=
Sent: Thursday, September 27, 2018 9:24 AM=0A=
To: Ferrell, Denise SDC=0A=
Cc: pgsql-admin@postgresql.org=0A=
Subject: Re: Data Corruption Errors=0A=
=0A=
On Thu, Sep 27, 2018 at 7:40 AM Ferrell, Denise SDC=0A=
<denise.ferrell@sdc-world.com> wrote:=0A=
=0A=
> Using Linux/PostgreSQL 9.5 &  setup on a VM.=0A=
=0A=
Please always show the entire version number -- it can matter a lot.=0A=
One way to see it is to run `select version();` on a database=0A=
connection.  From the shell you can run one of the executables with=0A=
the --version option, which will show the version and do nothing else.=0A=
If you are running in a VM, please say which product, and show the=0A=
exact version of that.=0A=
=0A=
> I'm receiving the following errors during an insert using a function but =
this does not happen every time.  When this does occur the database/system =
is unavailable until the processes are killed or the service is restarted.=
=0A=
>=0A=
> "LOG:  server process (PID xxxxx) was terminated by signal 9:  Killed=0A=
> DETAIL: Failed process was running=0A=
>    select insert_data ($1, $2,....)=0A=
> WARNING:  terminating connection because of crash of another server proce=
ss=0A=
> DETAIL:  The postmaster has commanded this server process to roll back th=
e current transaction and exit, because another server process exited abnor=
mally and possibly corrupted shared memory.=0A=
> HINT:  In a moment you should be able to reconnect ....=0A=
> LOG:  connection received=0A=
> FATAL:  the database system is in recovery mode"=0A=
>=0A=
> The last 2 lines are repeated and logging is not available.=0A=
>=0A=
> How can I determine data/table/shared memory corruption?=0A=
=0A=
You have not shown any evidence of corruption.  What happened was that=0A=
some database service process was killed with SIGKILL (kill -9).=0A=
Never do that.  It may have been done by the Out of Memory Killer=0A=
process.  You should disable that by setting vm.overcommit_memory=3D2=0A=
and vm.overcommit_ratio=3D80 *or maybe as high as 90).=0A=
=0A=
When a database service process is killed with SIGKILL the service=0A=
cannot continue without *RISK* of corruption, so it does a crash and=0A=
restart.  These messages:=0A=
=0A=
  FATAL:  the database system is in recovery mode=0A=
=0A=
Just mean that an attempt was made to the database while the recovery=0A=
was in progress, so the connection attempt was rejected.  Just ignore=0A=
those messages.  Just wait for the service to restart.=0A=
=0A=
--=0A=
Kevin Grittner=0A=
VMware vCenter Server=0A=
https://www.vmware.com/=0A=