Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1r1Opi-00Ayw0-Ll for pgsql-admin@arkaria.postgresql.org; Fri, 10 Nov 2023 10:33:51 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1r1Opg-002wSQ-Ei for pgsql-admin@arkaria.postgresql.org; Fri, 10 Nov 2023 10:33:48 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1r1Ope-002wSH-Uu for pgsql-admin@lists.postgresql.org; Fri, 10 Nov 2023 10:33:48 +0000 Received: from sonic307-53.consmr.mail.ir2.yahoo.com ([87.248.110.30]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1r1OpX-004tSG-2X for pgsql-admin@lists.postgresql.org; Fri, 10 Nov 2023 10:33:45 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.co.uk; s=s2048; t=1699612415; bh=lNJMyIFUzf+qNfJ8KZj821JfSPF88kS0WUad8eDjtbU=; h=Date:To:From:Subject:References:From:Subject:Reply-To; b=cXNG2I1Vqr9UZtJPKlj97ax06jr8umTS+jveXS/xGGcCz+NOixUOD3dDC7FpzHw/QeAZyhgeFkC7+M3KvhZPEAHQJKM2TIlV/Ww9iDBhsfwfDgSUyU/kLvx7G7FdVeoMPibAtJWc4bS8/D/rvNiEMkJtxy6ZfyQsODFoqKDwNTWsBwOe4sbjUR6EBSykk4OwhHTwIlKQ4bWWRap5vvG7kGMUn/yddt9HO2dImYMr0BOrhJDwOFQsDsrJW8pkPcB8Oy3o0Ww3ZyEcSE2sBS9NvfacGLCiQ4bNY2g36+6Hh3Bmqbs0aaU5EwFU/60kppm5NZ4m/YBbB54NIyUCrbpBww== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1699612415; bh=XGfgfLYq5059YVOHxcyKmMmAYwD9Wh9CcIeksThBtih=; h=X-Sonic-MF:Date:To:From:Subject:From:Subject; b=lp61fCgiJiUDKguaowkVShzGW/Oz1eLLxC+BUMDvVswH7YL2EW11RA2uMvYJZ4/FfleSU2pU/QMPhYZ7izMtY9Zi5UtkZ0ajUCL+P090b0/rpfookewZ+DrTcSBgAieZXo8QT5JSXtuXRUdAVk8i3T0zNjZdhkHIvnlZkq5hvjJpVS7vqA4kpBikQVdk0NTNbSIVvSO64Gs0cPwp6JG9GH8xET790XMgO/w6HxAys6ZI8gEnwLscKJmApisr1YJUS+r8BLWhCriX28KrAzP9shW7LmSLH2DYinf7+3MExvah9pRje/QGaZemQqqu/IltRr2TnJ5eppvhpLVbBPSSvQ== X-YMail-OSG: PkGcKHUVM1mS2mk77f1Vf3GHRtlrrVcCpXDZ1fDkCChGzGyuhgZ2ENYlcnre_o3 zgkxLvCMDmnjRV0F6UimWEfKuVQhjD3ck7v0FdlvY..0igRE9uTmqb77qy06TOVSNLSvZU97LY0u azapKgyD_0PJKMrx3ezWFCwHIM_pDgNfjMMF5ROX549mvuGrZ5WE5HUQl02J4Gt0wUWQ_qAqIzAw YTRRQTZnKZY6NKAnx8bAvXV96ZbxRy2PXXUojIX_ettYTXj_UvyC04UwhGixnqfDNXWDm5IUKHWA qMX3tP8dfQKcMfvIfbiBFQId4jGo4b.0wxWqCzKf2yIEWyry.ZvqPJxMarE5Rr8rdA3l4BEoWRpI a8ULarGoul_WDmJ4SNumQHmvq3pgaLZjds8p83YNfKaVEiBOrNcDqmpEQwXiUvI6eJ4DbSQnltzF M_V78mG9.nRaqW5.FR27HAeH_.8dcVEhb3e5.j43.F0Su2qCXrkiScrjWCZqX7_D7cJJQ5apP9zI Nn__fIU4vXI2uQtdXvr7SGowDcQ0BzXJpZfv4JfOLnHWcQyLRhqxulWyrfTzIF8BmjeHxR1RW6nd gHAL_GfX0BrUewdtmPN_Ry7VCvKaKnUkIv23bfmZeUwbe3OELCsxj5QjLKHU8JdSJ0pCHhv_9WFQ kwkSKIAnRFezxsmQ6sJrz4BzQEwDe8VfLNzEzWjB3Gdd4szNQQyYGokEUwFtyFAXDHy16RTzmmYX 0XPZIin9SGT1dBs5_nDAasTk.meSMZp1hCJx_bRJA_WX7wCswrqPfmF.j0s6yenP6yuxF97Z2Psc WPxlNzi8Rqy6CPtXQMl0qa_UGKu8.QkEu61WqGV_I3nHbOVjUbf2oSRlcfcdKZL0ch1ejaB_26gz oCz55eJyb_7V3i5XAuzvFSpzvJ4O_zD0XYZW1JgoLmbIaHtoyKuuFT2rq4Pdrlhh7ghY.OJcdUgr FQ4gpNQicZnU1SUwGeCs6ygpyGJnWJ2cLmywVlNNjzyXroHyokBgI3Wz3jFqADIVesa3vEc0ND9X UGxryRC2zcSJ.aIKJpbDjWD4D6WC5397HfoYEGCpblUAx0mMPANq4NaQEPGNsskqFvScDNoKP_g3 0q44cYQACHOydkCWGhKLv5azG036tN_vhqnha..g9Cbi.kg8yl6RN9B9Nd78AaiZZSd5T.2M._cf IQoc3IhjoZfUvpcpGiLLqKiLOXIIbMhlZd.FjDEHycsKQ5Q9c1jwjhE02mxQ8rxl2ivzNjJcMw05 ejCVqzHZCTA1PJnTxrt1Ij38GS8arTVI2G6r9ioTn.S9BU7MBGuwynWXqoi94brt5brbbTxqYVpB aC1Ni18O9Dk3hhescXD2x6YcK7w_nKwxSrcCiNl0dWUcoC.4bwwFs.ZfYM5mdMnBahbRppy_z8ey JBdjxo5h28HLbvIeHTnmLvUBs.Wh0rhGVt1ioe2iTfLtyN6f3fPIEYtfVPFSJpAjBr4WwyuNljkZ jz5.HlcM1s9_2PTG2uuv44bq.3wg1GmwPf34VBOEJJRH_9meIEARzS8kPEz46B3A4jgYr0rb8C06 Um5ZTDxdXwGmYdrh4wrsFTu72i00h_DbTuLcANgu_2DQ.OuG6cfn2Q8RkgNvgAY0xInU1LGg.VkD KLGbHCgeolkuIzy3was_k28AFR7XALQbD2n3OdufWM8gWBOa1TisuQ1OMI6Z.eUIA6R4W399N4Ag tLOtvJ7z2FDwpHl.8nyVeHv2z.0LCdw3.zeL88_bxPETu1NBLdnKGPwQa8LytRRXIuGJNn6XKSc1 aASX2WADib89CQCC_dqWbG0YnfAi9f8l8lMLhBQt5J3helaRTNgFCCQr7I.__7pQcFrPo0oa0Prv BoNHXMD5j_Iqyt3uE3xF6K6yKSBPXssabDQGDti8jN1eLn3SRiiidP0Cz_0Y1C6jHwUWbreQNeeC hOdlXWI50wQlg7GOt9xzXC5HKjZjWgQrznqH9fD47qYLZxJmWbP0T7yfIu1DrJyqgyk2sDz63dwq fTbbZhaeMTiV0wc.m1gzzS_jZ44KRB30SR1CxZ1nJwGnTo5lHlqzNh_LCX42URV6LdOa9C_gx_Rv lCeK.9h8094Gi_f2_rzklC7xAyXuG5aKpAmkwnnMAz4XQGeL1Co1z96JZ3XOYHEwitpShyTjvqFU yeZKpyyIUPGXELNNbNwk5skvXdByPNeRuxLybjrW_Ki7gWDOzJdQqKfSbKH_J7QCdRE2xwC.JK0. u7xgeFv.ZyLP8.a38YL6C6osNlNLkd6t9wMkTC6L3KlFV4MPwPxTdzKPvqefRzqz6I4zmrKkAWeW fDAMbT6tZbg-- X-Sonic-MF: X-Sonic-ID: 6a2822b8-9cfb-4c2d-9f9f-5aefd5d664d7 Received: from sonic.gate.mail.ne1.yahoo.com by sonic307.consmr.mail.ir2.yahoo.com with HTTP; Fri, 10 Nov 2023 10:33:35 +0000 Received: by hermes--production-ir2-6755f9ff7c-xgtp4 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 12b760ef6e83aac1169ceebc33386b6f; Fri, 10 Nov 2023 10:33:32 +0000 (UTC) Content-Type: multipart/alternative; boundary="------------ZtfUHmiikln1G0LuKeWFIsTk" Message-ID: <3c1964cc-ac89-4378-9b4f-841d8cb0fe01@yahoo.co.uk> Date: Fri, 10 Nov 2023 11:33:31 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: pgsql-admin@lists.postgresql.org Content-Language: en-US From: lejeczek Subject: fails to start - pg_stat_tmp ? References: <3c1964cc-ac89-4378-9b4f-841d8cb0fe01.ref@yahoo.co.uk> X-Mailer: WebService/1.1.21896 mail.backend.jedi.jws.acl:role.jedi.acl.token.atz.jws.hermes.yahoo Content-Length: 5092 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk This is a multi-part message in MIME format. --------------ZtfUHmiikln1G0LuKeWFIsTk Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Hi guys. I have simple cluster managed by a popular toolset, I believe, and I get a standby/slave node which after it was shutdown, rebooted, is unable to start, not by the that cluster-manager nor manually. I get: ... 2023-11-10 10:18:45.491 UTC [54858] LOG:  starting PostgreSQL 14.9 (Ubuntu 14.9-0ubuntu0.22.04.1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, 64-bit 2023-11-10 10:18:45.491 UTC [54858] LOG:  listening on IPv4 address "0.0.0.0", port 5433 2023-11-10 10:18:45.491 UTC [54858] LOG:  listening on IPv6 address "::", port 5433 2023-11-10 10:18:45.495 UTC [54858] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5433" 2023-11-10 10:18:45.505 UTC [54860] LOG:  database system was interrupted while in recovery at log time 2023-11-10 09:53:18 UTC 2023-11-10 10:18:45.505 UTC [54860] HINT:  If this has occurred more than once some data might be corrupted and you might need to choose an earlier recovery target. 2023-11-10 10:18:45.540 UTC [54862] drwszelakiadmmdisc@drwszelaki_discourse FATAL:  the database system is starting up 2023-11-10 10:18:45.545 UTC [54863] drwszelakiadmmdisc@drwszelaki_discourse FATAL:  the database system is starting up 2023-11-10 10:18:45.795 UTC [54860] LOG:  entering standby mode 2023-11-10 10:18:45.799 UTC [54860] FATAL:  could not open directory "/var/run/postgresql/14-paf.pg_stat_tmp": No such file or directory 2023-11-10 10:18:45.800 UTC [54858] LOG:  startup process (PID 54860) exited with exit code 1 2023-11-10 10:18:45.800 UTC [54858] LOG:  aborting startup due to startup process failure 2023-11-10 10:18:45.802 UTC [54858] LOG:  database system is shut down Indeed that dir, that path does not exist on the node - from what I can see, among the nodes comprising the cluster this path exists only on the _master_ node. What would that specific errors be suggesting or meaning - do I have something mis-configured, perhaps the toolset itself does not deal with "mere" reboot well, or at all. Toolset is HA/pcs(pacemaker) cluster tools. many thanks, L. --------------ZtfUHmiikln1G0LuKeWFIsTk Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit Hi guys.

I have simple cluster managed by a popular toolset, I believe, and I get a standby/slave node which after it was shutdown, rebooted, is unable to start, not by the that cluster-manager nor manually.
I get:
...
2023-11-10 10:18:45.491 UTC [54858] LOG:  starting PostgreSQL 14.9 (Ubuntu 14.9-0ubuntu0.22.04.1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, 64-bit
2023-11-10 10:18:45.491 UTC [54858] LOG:  listening on IPv4 address "0.0.0.0", port 5433
2023-11-10 10:18:45.491 UTC [54858] LOG:  listening on IPv6 address "::", port 5433
2023-11-10 10:18:45.495 UTC [54858] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5433"
2023-11-10 10:18:45.505 UTC [54860] LOG:  database system was interrupted while in recovery at log time 2023-11-10 09:53:18 UTC
2023-11-10 10:18:45.505 UTC [54860] HINT:  If this has occurred more than once some data might be corrupted and you might need to choose an earlier recovery target.
2023-11-10 10:18:45.540 UTC [54862] drwszelakiadmmdisc@drwszelaki_discourse FATAL:  the database system is starting up
2023-11-10 10:18:45.545 UTC [54863] drwszelakiadmmdisc@drwszelaki_discourse FATAL:  the database system is starting up
2023-11-10 10:18:45.795 UTC [54860] LOG:  entering standby mode
2023-11-10 10:18:45.799 UTC [54860] FATAL:  could not open directory "/var/run/postgresql/14-paf.pg_stat_tmp": No such file or directory
2023-11-10 10:18:45.800 UTC [54858] LOG:  startup process (PID 54860) exited with exit code 1
2023-11-10 10:18:45.800 UTC [54858] LOG:  aborting startup due to startup process failure
2023-11-10 10:18:45.802 UTC [54858] LOG:  database system is shut down


Indeed that dir, that path does not exist on the node - from what I can see, among the nodes comprising the cluster this path exists only on the _master_ node.

What would that specific errors be suggesting or meaning - do I have something mis-configured, perhaps the toolset itself does not deal with "mere" reboot well, or at all.
Toolset is HA/pcs(pacemaker) cluster tools.

many thanks, L.
--------------ZtfUHmiikln1G0LuKeWFIsTk--