Received: from malur.postgresql.org ([2a02:16a8:dc51::56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1g2x6D-0002tW-5O for pgsql-admin@arkaria.postgresql.org; Thu, 20 Sep 2018 11:26:22 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1g2x6B-00059w-9N for pgsql-admin@arkaria.postgresql.org; Thu, 20 Sep 2018 11:26:19 +0000 Received: from makus.postgresql.org ([2001:4800:1501:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1g2x6A-00059k-FB for pgsql-admin@lists.postgresql.org; Thu, 20 Sep 2018 11:26:19 +0000 Received: from mail-oln040092253043.outbound.protection.outlook.com ([40.92.253.43] helo=APC01-SG2-obe.outbound.protection.outlook.com) by makus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1g2x65-0005MV-Vw for pgsql-admin@postgresql.org; Thu, 20 Sep 2018 11:26:17 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5HG24rtIPJhwUIR2AaLRQ0vHIwdQf+qag/E9JjBx7K4=; b=Eoz2am819UBhw75WoN03K4DivHAmWyZ+DzLYlLPJ0ILfEMO8ao+5+GqmKUDWcsP8dpLP4X12nGpPyGDLjJnzyAzsUFQgJqETuEBR7JLyPLviWZioRmv9XfteJvz/nPX4q6pELadmkkpNrsHwKTivP8zqutmuJ6PlCFyfqR8+a3xQ4How4EMUfdCc8wCcKcbBH3LaAOoPzF92ps3iahJaDlzmPMYTFwqs1v/TqgwcWQ5d3UJNcKHC8n1C2AVK8k6ER+TYOpNi9wL8aZ37VK+sS5y9Br7tameVe0qDEQtujFHmm0+T5hnxBoxGfKN7k9LWYUTN0AvQkZ7hDNBXkI0mMA== Received: from HK2APC01FT031.eop-APC01.prod.protection.outlook.com (10.152.248.56) by HK2APC01HT106.eop-APC01.prod.protection.outlook.com (10.152.249.166) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.20.1164.13; Thu, 20 Sep 2018 11:26:09 +0000 Received: from MA1PR0101MB1687.INDPRD01.PROD.OUTLOOK.COM (10.152.248.56) by HK2APC01FT031.mail.protection.outlook.com (10.152.248.189) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.20.1164.13 via Frontend Transport; Thu, 20 Sep 2018 11:26:09 +0000 Received: from MA1PR0101MB1687.INDPRD01.PROD.OUTLOOK.COM ([fe80::851c:4cdc:a11f:d54b]) by MA1PR0101MB1687.INDPRD01.PROD.OUTLOOK.COM ([fe80::851c:4cdc:a11f:d54b%3]) with mapi id 15.20.1143.017; Thu, 20 Sep 2018 11:26:09 +0000 From: soumitra bhandary To: arvind chikne CC: "thomas.poty@gmail.com" , pgsql-admin Subject: Re: Master server stoped working Thread-Topic: Master server stoped working Thread-Index: AQHUUMIBcVXtB/4uhkW7wbtAAkQD06T45YuAgAACbQCAAA5Lr4AABWYAgAAG+OmAAANcAIAAAOzZ Date: Thu, 20 Sep 2018 11:26:09 +0000 Message-ID: References: , In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-incomingtopheadermarker: OriginalChecksum:B601319DC5D472009BC79E4DAFE64ADA823BE48D093127B441BDE61ED8859C38;UpperCasedChecksum:38662C8E966C15ED382853C0AFCC5E265A2F00E740231060A41A374F6AFFB7C7;SizeAsReceived:7647;Count:47 x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [vAShd+JySxebLJ/OJ2R+J52DSZSXZ71+] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;HK2APC01HT106;6:WrIST4f5SOUqK7WtohHqT+oOTHE47ijNF3uJhVjyRWVBjlsyGV9FY9RQzmNdrdbgFThbg3SygKKLmQhGcoe3zd4oR38z4kWaabvohT27RWhFT4YRGGzrKbkBTh20AY8z4R+2IPdxmxAE/e+ImsOC926IspcjhBIYFO1mqweZqa0K2+5k+RidYzgGQMSk1AV0js4RUctRQXy4QeDYa3PCivnCDheHc0jGkNE8ZEP2iaJpTs7WlirXy8IzPzxu88dsa6nLbPknf2nCDlxbo4ZuTMM8e9RlVCfDNLg1RcPCk1cHFzwC9kx12VdHMYRuFuFH4Lq9lcyA7VwU1xFtibCsP9dj0Wlz/MQKsVmXkckPSeVhUXpHvWdTPOMh0XUXY63orD7aHMwNxKzsMAMqnR4ZzQwwM6APFxJdKx4MxVscfvaHILHRaqo/NQVlZ0wbk1MxOetCM8s3Rvdt20kXZGwjPQ==;5:xwtJcrlMuQ+bS7ULVVsi7z5s2N2smv6IilRHikMw7jeOP2yfiuZu5AorIktlo0hVpdU8gLRUx1TaS75K3XpM4kpeERyMlDlr+bfrFIkZ9QOYdEU48XQyl2+JLrRPrSa4Ea4Zd19QGGI6T10sTRIXd5fgv88mhtlFo3bR5BQkzdU=;7:0cXjtOz6PU4Ika6eP9dIBVoEk1qUSTTNTXF4lKnzho77rzVsqrdKHHZtn8lCjYiqJ6ws486ZcebO1d7wVXPEaOFNd4+aiVtrM+DbeTTGZWZOo9USMe3MxxqoZdYLwF61Hd7QIbXk3K/o6yeY7cSuu0UMhvNTPUuXVjUAO4rMqeZ0N9RCj/FmrUjmOoUWa+NHeEgO8uOwbXQnCXHpMxOOPEVzMnXQCGUahlflxC6kDx+ajc4km58GUgiH9fSLAvvv x-incomingheadercount: 47 x-eopattributedmessage: 0 x-microsoft-antispam: BCL:0;PCL:0;RULEID:(7020095)(201702061078)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031324274)(2017031323274)(2017031322404)(1601125500)(1603101475)(1701031045);SRVR:HK2APC01HT106; x-ms-traffictypediagnostic: HK2APC01HT106: x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(4566010)(82015058);SRVR:HK2APC01HT106;BCL:0;PCL:0;RULEID:;SRVR:HK2APC01HT106; x-forefront-prvs: 0801F2E62B x-forefront-antispam-report: SFV:NSPM;SFS:(7070007)(199004)(53754006)(189003)(51444003)(105586002)(106356001)(45080400002)(14444005)(87572001)(86152003)(54906003)(73972006)(68736007)(102836004)(486006)(236005)(5003630100001)(256004)(2900100001)(99286004)(71200400001)(9686003)(71190400001)(97736004)(6506007)(86362001)(53546011)(76176011)(74316002)(5250100002)(3480700004)(7696005)(55016002)(56003)(8936002)(25786009)(19627405001)(6436002)(6246003)(83332001)(93886005)(104016004)(4326008)(54896002)(551934003)(39060400002)(14454004)(8676002)(33656002)(6916009)(105004)(229853002)(82202002)(476003)(6346003)(20460500001)(5660300001)(446003)(11346002)(26005)(81156014)(15852004);DIR:OUT;SFP:1901;SCL:1;SRVR:HK2APC01HT106;H:MA1PR0101MB1687.INDPRD01.PROD.OUTLOOK.COM;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: hotmail.com does not designate permitted sender hosts) authentication-results: spf=none (sender IP is ) smtp.mailfrom=soumitra.bhandary@hotmail.com; x-microsoft-antispam-message-info: O6yQBysZ5WaiX/sSS6ZPtELLBvQut6HCIQR27KxLNSfih3qkSOgvjHU58XBCLnJJEWR61dhdodSTHEcfGMO6mCxCuQk7tct4hvB5t8XIhMvOTdOzA+nKba4EbNYlTKsM8borWP+taXyXJxCqgUxL8v4g1CsiAhdKDTm7qUD2mPjd4SOeb7nwpsz+rqGjwXLRSIAJgbCXWtZn84R78eqDUhVfrXwhRRNP2J0iOYvfvZs= Content-Type: multipart/alternative; boundary="_000_MA1PR0101MB1687448BAF0EBB8AA0F84CC3E0130MA1PR0101MB1687_" MIME-Version: 1.0 X-OriginatorOrg: hotmail.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 7181d4b0-87d6-4f4e-ba33-0d3746212cec X-MS-Exchange-CrossTenant-Network-Message-Id: 5eba1121-c068-4b23-c881-08d61eebdcfc X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 7181d4b0-87d6-4f4e-ba33-0d3746212cec X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Sep 2018 11:26:09.2835 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: HK2APC01HT106 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --_000_MA1PR0101MB1687448BAF0EBB8AA0F84CC3E0130MA1PR0101MB1687_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable It's my bad . Have you been executed from /usr/lib/postgresql/9.5/bin directory where pg_= ctl utility is present . or you can execute /usr/lib/postgresql/9.5/bin/Pg_ctl start -D {PGDATA} {PGDATA} :: To be replaced with data directory where configuration file is = present and Log in user must be postgres . Thanks, Soumitra ________________________________ From: arvind chikne Sent: Thursday, September 20, 2018 4:49 PM To: soumitra.bhandary@hotmail.com Cc: thomas.poty@gmail.com; pgsql-admin Subject: Re: Master server stoped working Hi pg_ctl: command not found On Thu, Sep 20, 2018 at 4:37 PM soumitra bhandary > wrote: Hi Arvind , Ok .. can you execute and share the output Pg_ctl start -D {PGDATA} Sent from my iPhone On 20-Sep-2018, at 4:12 PM, arvind chikne > wrote: Hi Soumitra, I think that's not the issue all the files are at the right place, it was w= orking fine till yesterday On Thu, Sep 20, 2018 at 3:55 PM soumitra bhandary > wrote: Hi Arvind , Can you please check the location of postgresql.conf file ? It looks like ,= it is in different location than what you have given in the command i.e. (= -D /var/lib/postgresql/9.5/main) Thanks , Soumitra ________________________________ From: arvind chikne > Sent: Thursday, September 20, 2018 3:01 PM To: thomas.poty@gmail.com Cc: pgsql-admin Subject: Re: Master server stoped working Actually its seems that there is no activity captured in the /var/log/post= gresql/postgresql-9.5-main.log pls see below the log root@Master:~# tail -f /var/log/postgresql/postgresql-9.5-main.log 2018-09-19 17:32:41 IST [325-9] dhis@up HINT: In a moment you should be ab= le to reconnect to the database and repeat your command. 2018-09-19 17:32:41 IST [31038-1] dhis@dhis WARNING: terminating connectio= n because of crash of another server process 2018-09-19 17:32:41 IST [31038-2] dhis@dhis DETAIL: The postmaster has com= manded this server process to roll back the current transaction and exit, b= ecause another server process exited abnormally and possibly corrupted shar= ed memory. 2018-09-19 17:32:41 IST [31038-3] dhis@dhis HINT: In a moment you should b= e able to reconnect to the database and repeat your command. 2018-09-19 17:32:41 IST [324-49] dhis@up WARNING: terminating connection b= ecause of crash of another server process 2018-09-19 17:32:41 IST [324-50] dhis@up DETAIL: The postmaster has comman= ded this server process to roll back the current transaction and exit, beca= use another server process exited abnormally and possibly corrupted shared = memory. 2018-09-19 17:32:41 IST [324-51] dhis@up HINT: In a moment you should be a= ble to reconnect to the database and repeat your command. 2018-09-19 17:32:41 IST [27616-3] dhis@up WARNING: terminating connection = because of crash of another server process 2018-09-19 17:32:41 IST [27616-4] dhis@up DETAIL: The postmaster has comma= nded this server process to roll back the current transaction and exit, bec= ause another server process exited abnormally and possibly corrupted shared= memory. 2018-09-19 17:32:41 IST [27616-5] dhis@up HINT: In a moment you should be = able to reconnect to the database and repeat your command. On Thu, Sep 20, 2018 at 2:53 PM Thomas Poty > wrote: Hi, Could you share your log in /var/log/postgresql/postgresql-9.5-main.log fro= m server start to the end ? Le jeu. 20 sept. 2018 =E0 11:11, arvind chikne > a =E9crit : Hi All of sudden my master server goes down pls see the error below when tried= to start it Sep 20 14:33:30 Master postgresql@9.5-main[2162]: Error: could not exec /us= r/lib/postgresql/9.5/bin/pg_ctl /usr/lib/postgresql/9.5/bin/pg_ctl start -D= /var/lib/postgresql/9.5/main -l /var/log/postgresql/postgresql-9.5-main.lo= g -s -o -c config_file=3D"/etc/postgresql/9.5/main/postgresql.conf" : Sep 20 14:33:30 Master systemd[1]: postgresql@9.5-main.service: Control process exited, code=3Dexited status=3D1 Sep 20 14:33:30 Master systemd[1]: Failed to start PostgreSQL Cluster 9.5-m= ain. Sep 20 14:33:30 Master systemd[1]: postgresql@9.5-main.service: Unit entered failed state. Sep 20 14:33:30 Master systemd[1]: postgresql@9.5-main.service: Failed with result 'exit-code'. -- Thanks & Regards Arvind rao chikne 9818979986 Think Green!!! Please consider your environmental responsibility before pri= nting this e-mail -- Thanks & Regards Arvind rao chikne 9818979986 Think Green!!! Please consider your environmental responsibility before pri= nting this e-mail -- Thanks & Regards Arvind rao chikne 9818979986 Think Green!!! Please consider your environmental responsibility before pri= nting this e-mail -- Thanks & Regards Arvind rao chikne 9818979986 Think Green!!! Please consider your environmental responsibility before pri= nting this e-mail --_000_MA1PR0101MB1687448BAF0EBB8AA0F84CC3E0130MA1PR0101MB1687_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
= It's my bad . 

= Have you been executed from /usr/lib/postgresql/9.5/bin directory where pg_= ctl utility is present . 

= or you can execute /usr/lib/postgresql/9.5/bin/Pg_ctl start -D {PGDATA}

{PGDATA} :: To be replaced with data directory where configuration file is present = and Log in user must be postgres .

Thanks, 
Soumitra 

From: arvind chikne <arv= ind.chikne@gmail.com>
Sent: Thursday, September 20, 2018 4:49 PM
To: soumitra.bhandary@hotmail.com
Cc: thomas.poty@gmail.com; pgsql-admin
Subject: Re: Master server stoped working
 
Hi 

pg_ctl: command not found


On Thu, Sep 20, 2018 at 4:37 PM soumitra bhandary <soumitra.bhandary@hotmail.com<= /a>> wrote:
Hi Arvind , 

Ok .. can you execute and share the output 

Pg_ctl start -D {PGDATA}

Sent from my iPhone<= /div>
Hi Soumitra, 

I think that's not the issue all the files are at the right place, it = was working fine till yesterday 

On Thu, Sep 20, 2018 at 3:55 PM soumitra bhandary <soumitra.bha= ndary@hotmail.com> wrote:
Hi Arvind , 

Can you please check the location of postgresql.conf file ? It looks like , it is in different location tha= n what you have given in the command i.e. ( -D /var/lib/postgresql/9.5/main)

Thanks , 
Soumitra 
From: arvind chikne <arvind.chikne@gmail.com>
Sent: Thursday, September 20, 2018 3:01 PM
To: thoma= s.poty@gmail.com
Cc: pgsql-admin
Subject: Re: Master server stoped working
 
Actually its seems that there is no activity captured in t= he  /var/log/postgresql/postgresql-9.5-main.log 
pls see below the log
root@Master:~# tail -f /var/log/postgresql/postgresql-9.5-main.log
2018-09-19 17:32:41 IST [325-9] dhis@up HINT:  In a moment you sh= ould be able to reconnect to the database and repeat your command.
2018-09-19 17:32:41 IST [31038-1] dhis@dhis WARNING:  terminating= connection because of crash of another server process
2018-09-19 17:32:41 IST [31038-2] dhis@dhis DETAIL:  The postmast= er has commanded this server process to roll back the current transaction a= nd exit, because another server process exited abnormally and possibly corr= upted shared memory.
2018-09-19 17:32:41 IST [31038-3] dhis@dhis HINT:  In a moment yo= u should be able to reconnect to the database and repeat your command.
2018-09-19 17:32:41 IST [324-49] dhis@up WARNING:  terminating co= nnection because of crash of another server process
2018-09-19 17:32:41 IST [324-50] dhis@up DETAIL:  The postmaster = has commanded this server process to roll back the current transaction and = exit, because another server process exited abnormally and possibly corrupt= ed shared memory.
2018-09-19 17:32:41 IST [324-51] dhis@up HINT:  In a moment you s= hould be able to reconnect to the database and repeat your command.
2018-09-19 17:32:41 IST [27616-3] dhis@up WARNING:  terminating c= onnection because of crash of another server process
2018-09-19 17:32:41 IST [27616-4] dhis@up DETAIL:  The postmaster= has commanded this server process to roll back the current transaction and= exit, because another server process exited abnormally and possibly corrup= ted shared memory.
2018-09-19 17:32:41 IST [27616-5] dhis@up HINT:  In a moment you = should be able to reconnect to the database and repeat your command.


On Thu, Sep 20, 2018 at 2:53 PM Thomas Poty <thomas.poty@gmail.com&= gt; wrote:
Hi,
Could you share your log in /var/log/postgresql/postgresql-9.5-main.lo= g from server start
 to the end ?

Le jeu. 20 sept. 2018 =E0 11:11, arvind chikne &= lt;arvind.chik= ne@gmail.com> a =E9crit :
Hi
All of sudden my master server goes down pls see the error below when = tried to start it 

Sep 20 14:33:30 Master postgresql@9.5-main[2162]: Error: could not exe= c /usr/lib/postgresql/9.5/bin/pg_ctl /usr/lib/postgresql/9.5/bin/pg_ctl sta= rt -D /var/lib/postgresql/9.5/main -l /var/log/postgresql/postgresql-9.5-ma= in.log -s -o  -c config_file=3D"/etc/postgresql/9.5/main/postgres= ql.conf" :
Sep 20 14:33:30 Master systemd[1]: postgresql@9.5-main.service: Control process exited, code=3Dexited stat= us=3D1
Sep 20 14:33:30 Master systemd[1]: Failed to start PostgreSQL Cluster = 9.5-main.
Sep 20 14:33:30 Master systemd[1]: postgresql@9.5-main.service: Unit entered failed state.
Sep 20 14:33:30 Master systemd[1]: postgresql@9.5-main.service: Failed with result 'exit-code'.


--
Thanks & Regard= s
Arvind rao chikne
9818979986=
Think Green!!!=  Please&= nbsp;con= sider yo= ur environmental responsibility before printing this e-mail



--
Thanks & Regard= s
Arvind rao chikne
9818979986=
Think Green!!!=  Please&= nbsp;con= sider yo= ur environmental responsibility before printing this e-mail



--
Thanks & Regard= s
Arvind rao chikne
9818979986=
Think Green!!!=  Please&= nbsp;con= sider yo= ur environmental responsibility before printing this e-mail



--
Thanks & Regard= s
Arvind rao chikne
9818979986=
Think Green!!!=  Please&= nbsp;con= sider yo= ur environmental responsibility before printing this e-mail

--_000_MA1PR0101MB1687448BAF0EBB8AA0F84CC3E0130MA1PR0101MB1687_--