BUG #17954: Postgres startup fails with `could not locate a valid checkpoint record`

pg.ddx.io  pgsql-bugs@postgresql.org mailing list archive  
help / color / mirror / Atom feed

BUG #17954: Postgres startup fails with `could not locate a valid checkpoint record`
2+ messages / 2 participants
[nested] [flat]

* BUG #17954: Postgres startup fails with `could not locate a valid checkpoint record`
@ 2023-06-01 13:11  PG Bug reporting form <noreply@postgresql.org>
  0 siblings, 1 reply; 2+ messages in thread

From: PG Bug reporting form @ 2023-06-01 13:11 UTC (permalink / raw)
  To: pgsql-bugs@lists.postgresql.org; +Cc: srivastavautkarsh8097@gmail.com

The following bug has been logged on the website:

Bug reference:      17954
Logged by:          Utkarsh Srivastava
Email address:      srivastavautkarsh8097@gmail.com
PostgreSQL version: 12.12
Operating system:   RHEL/Linux
Description:        

Hi everyone,

Thank you for your time. We are running PostgreSQL 12.12 in a CRI-O
container on top of CephFS. A few days ago we noticed that DB startup was
failing with the following error:
```
2023-05-14 05:13:13.678 UTC [1] LOG:  received smart shutdown request
2023-05-14 05:13:36.692 UTC [1] LOG:  could not open file "postmaster.pid":
No such file or directory
2023-05-14 05:13:36.692 UTC [1] LOG:  performing immediate shutdown because
data directory lock file is invalid
2023-05-14 05:13:36.692 UTC [1] LOG:  received immediate shutdown request
2023-05-14 05:13:36.692 UTC [1] LOG:  could not open file "postmaster.pid":
No such file or directory
2023-05-14 05:13:36.692 UTC [261282] WARNING:  terminating connection
because of crash of another server process
2023-05-14 05:13:36.692 UTC [261282] DETAIL:  The postmaster has commanded
this server process to roll back the current transaction and exit, because
another server process exited abnormally and possibly corrupted shared
memory.
2023-05-14 05:13:36.692 UTC [261282] HINT:  In a moment you should be able
to reconnect to the database and repeat your command.
< --- Trimmed repetition ---> 
2023-05-14 05:13:36.739 UTC [1] LOG:  database system is shut down
2023-05-14 05:13:37.723 UTC [24] LOG:  database system was shut down at
2023-05-14 05:13:17 UTC
2023-05-14 05:13:37.723 UTC [24] LOG:  invalid resource manager ID 101 at
9/8BF289E8
2023-05-14 05:13:37.723 UTC [24] LOG:  invalid primary checkpoint record
2023-05-14 05:13:37.723 UTC [24] PANIC:  could not locate a valid checkpoint
record
2023-05-14 05:13:39.961 UTC [22] LOG:  startup process (PID 24) was
terminated by signal 6: Aborted
2023-05-14 05:13:39.961 UTC [22] LOG:  aborting startup due to startup
process failure
2023-05-14 05:13:40.117 UTC [22] LOG:  database system is shut down
2023-05-14 05:14:06.726 UTC [24] LOG:  database system was shut down at
2023-05-14 05:13:17 UTC
2023-05-14 05:14:06.726 UTC [24] LOG:  invalid resource manager ID 101 at
9/8BF289E8
```

- What could be the root cause of this issue? 
- Is this a known issue (I did search the archives but couldn't find it
though)? If yes, is this fixed in a PG 13, 14, 15?

Thank you

^ permalink  raw  reply  [nested|flat] 2+ messages in thread

* Re: BUG #17954: Postgres startup fails with `could not locate a valid checkpoint record`
@ 2023-06-12 04:56  Michael Paquier <michael@paquier.xyz>
  parent: PG Bug reporting form <noreply@postgresql.org>
  0 siblings, 0 replies; 2+ messages in thread

From: Michael Paquier @ 2023-06-12 04:56 UTC (permalink / raw)
  To: srivastavautkarsh8097@gmail.com; pgsql-bugs@lists.postgresql.org

On Thu, Jun 01, 2023 at 01:11:20PM +0000, PG Bug reporting form wrote:
> - What could be the root cause of this issue? 
> - Is this a known issue (I did search the archives but couldn't find it
> though)? If yes, is this fixed in a PG 13, 14, 15?

Hard to say for sure, but it looks like your host has a few problems.
This part from your logs refers to something that should not happen,
to begin with:

> 2023-05-14 05:13:13.678 UTC [1] LOG:  received smart shutdown request
> 2023-05-14 05:13:36.692 UTC [1] LOG:  could not open file "postmaster.pid":
> No such file or directory
> 2023-05-14 05:13:36.692 UTC [1] LOG:  performing immediate shutdown because
> data directory lock file is invalid
> 2023-05-14 05:13:36.692 UTC [1] LOG:  received immediate shutdown request
> 2023-05-14 05:13:36.692 UTC [1] LOG:  could not open file "postmaster.pid":
> No such file or directory

This LOG would come from either AddToDataDirLockFile() or
RecheckDataDirLockFile().  Still, the third entry I am quoting refers
to a recheck of the PID file, meaning that the postmaster has bumped
into what looks like a corrupted PID file.  
--
Michael

Attachments:

  signature.asc (application/pgp-signature, 832B) download

^ permalink  raw  reply  [nested|flat] 2+ messages in thread

end of thread, other threads:[~2023-06-12 04:56 UTC | newest]

Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2023-06-01 13:11 BUG #17954: Postgres startup fails with `could not locate a valid checkpoint record` PG Bug reporting form <noreply@postgresql.org>
2023-06-12 04:56 ` Michael Paquier <michael@paquier.xyz>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox