Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1sxTZ8-00AM8t-Cq for pgsql-general@arkaria.postgresql.org; Sun, 06 Oct 2024 15:53:02 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1sxTZ5-00GC0L-OU for pgsql-general@arkaria.postgresql.org; Sun, 06 Oct 2024 15:52:59 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1sxTZ4-00GBsw-76 for pgsql-general@lists.postgresql.org; Sun, 06 Oct 2024 15:52:59 +0000 Received: from fhigh-a4-smtp.messagingengine.com ([103.168.172.155]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1sxTYv-002snP-NF for pgsql-general@postgresql.org; Sun, 06 Oct 2024 15:52:56 +0000 Received: from phl-compute-10.internal (phl-compute-10.phl.internal [10.202.2.50]) by mailfhigh.phl.internal (Postfix) with ESMTP id ADFA811400DA; Sun, 6 Oct 2024 11:52:47 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-10.internal (MEProxy); Sun, 06 Oct 2024 11:52:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aklaver.com; h= cc:cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm1; t=1728229967; x=1728316367; bh=gUimPVEAG278fh4UtQMdoLkudh2YfGldYP/F5/eRSyY=; b= tfBEKJp5hPvxZlGBSqjYGkXukUDdKtU+lWPGAHBSAAcVzJpWTlpSvboUVUNz+FNj J1G937wvqr4EFmDiyndbFBclXi7m58Omh7WRH81ry+ci0zpzWnYkX79w09IAfmoj besRhMSf7UZ/zVdsdo2aadHtEEUlJj3UdShU/TaqXNj9uFFMQWHV/4swjroj6/px dv/fuae6nOdPEJDpwyG4uo1e8u+QhtKoJoSbwkgub0b/E5+PmSiMD8mJfk8FhFBM +exaKBZkjsKYzBOdFQRyhAwXsqBCOc5eHWH/J/fv5oE+i/PmaJum7oYbjCAMhQLN +ri9L0G14ONzlT6EFF3aWA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1728229967; x= 1728316367; bh=gUimPVEAG278fh4UtQMdoLkudh2YfGldYP/F5/eRSyY=; b=J 5FhE/DQiPI9gRZs/q3h6cdwG75Zy8xZKGWNZVFTcGI+MgArQjQzmUOeVH1qyaRWv pX3vfVoqplnjdGCpw3gI4vaedwQqJPoBnUqVbm/NJx5NGtyXm6vFMWpmTaiTBImA kOXV/qhOtuYOujJnsSGaEk142hVmZcuV415HSolv8kfxeBJbaNqO2XMBn9pgQnrn XwObFTgpr/ez+yAFDuAuSc6at4uLgUakpW/fgBAD1IMZ6Av5vBC3mSmbLJZRg0IV f7IYXtcXW6LMkhrNAAahfuCH53QfmOIxTPYRqWYm1mySpGVVlJELB2I8dA5IMyFK 6FiXI5DCYt6vgqDzxoZ8w== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrvddvjedgleegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucenucfjughrpefkffggfg fuvfevfhfhjggtgfesthekredttddvjeenucfhrhhomheptegurhhirghnucfmlhgrvhgv rhcuoegrughrihgrnhdrkhhlrghvvghrsegrkhhlrghvvghrrdgtohhmqeenucggtffrrg htthgvrhhnpeefgeefieeutdfggfetgefgheekjeehteeileeigfetieekjedvieeviefg heevtdenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe grughrihgrnhdrkhhlrghvvghrsegrkhhlrghvvghrrdgtohhmpdhnsggprhgtphhtthho pedvpdhmohguvgepshhmthhpohhuthdprhgtphhtthhopehkkhgthhhnrdhinhesghhmrg hilhdrtghomhdprhgtphhtthhopehpghhsqhhlqdhgvghnvghrrghlsehpohhsthhgrhgv shhqlhdrohhrgh X-ME-Proxy: Feedback-ID: i76984098:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sun, 6 Oct 2024 11:52:46 -0400 (EDT) Message-ID: <6ca1ca9f-fdcf-436c-bf2a-800107f6004b@aklaver.com> Date: Sun, 6 Oct 2024 08:52:46 -0700 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: CLOSE_WAIT pileup and Application Timeout To: KK CHN Cc: pgsql-general References: Content-Language: en-US From: Adrian Klaver In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On 10/6/24 06:26, KK CHN wrote: > > > On Fri, Oct 4, 2024 at 9:17 PM Adrian Klaver Seems the issue is in the application server. What is not clear to > me is > whether the connection timeout you refer to is from the mobile devices > to the application or the application to the Postgres server? > > its from mobile devices to application server.  When I do a restart of > application server everything backs to normal.  But after a period of > time again it cripples.  That time when I netstat on Application VM lots > of  CLOSE_WAIT states as indicated. > > I'm > guessing the latter as I would expect the mobile devices to drop > connections more often then weekly. > > Yes mobile devices may drops connections at any point of time if it > reaches an area where signal strength is poor( eg; Underground > parking or near the areas where mobile data coverage is poor. > > > > > The topology is mobile devices  connect and update the location via > application VM then   finally in  PGSQL VM. > > The application server and  Database server both separate virtual > machines.      Application server hangs most often not the database VM. > Since there are other applications which update to the database VM > without any issue.  The DB VM caters all the writes from other > applications. But those applications are different, not fleet management > one. From what I see this really has nothing to do with the Postgres backend. It is a matter of communication, actually lack of communication, between the mobile devices and the application server. A broad answer is that something needs to be done to gracefully deal with mobile device connection drops -- Adrian Klaver adrian.klaver@aklaver.com