Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1tAOBi-00FSZx-HY for pgsql-admin@arkaria.postgresql.org; Mon, 11 Nov 2024 06:46:14 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1tAOBe-00Bw3u-9y for pgsql-admin@arkaria.postgresql.org; Mon, 11 Nov 2024 06:46:10 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1tAOBd-00Bw26-OG for pgsql-admin@lists.postgresql.org; Mon, 11 Nov 2024 06:46:10 +0000 Received: from mail-pg1-x535.google.com ([2607:f8b0:4864:20::535]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1tAOBb-001Flk-LL for pgsql-admin@lists.postgresql.org; Mon, 11 Nov 2024 06:46:09 +0000 Received: by mail-pg1-x535.google.com with SMTP id 41be03b00d2f7-7eda47b7343so2759507a12.0 for ; Sun, 10 Nov 2024 22:46:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731307566; x=1731912366; darn=lists.postgresql.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=SDVY5jDC/lD39KR3EwR8FWHntl1FUssp0GwcmgGdFZA=; b=CxnGdJN0oMKyNBk8wT+uSjYWrf9wVhx4pdwaeD11Pvn8XeWNYcsVpVBvzRHJnCgqU2 oyzYftVpA68bLmc9w2pg1UInEQdW/MHRfm4J284eVo8YuamrfKy36Mg/1jn9gKDtH+lp HqsiehJvKCZHkseM11jqh3V23EmNufn96U+ep0ihvLUZPLAS8hd12fz6B8yDnmCSS1+6 C2joXYfe6yYlsrhYfqd9/hTs5T6e1aQSiaiGEHGIPf1dxzLpHaY79/chxowhlPPWgQ7A Srwzhs3tEB1vev81a0rfYgLYbwFgdqvKCIpYjeycHgMJhbNpL+JOJ56HBXSuUi+RfAMC GinA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731307566; x=1731912366; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=SDVY5jDC/lD39KR3EwR8FWHntl1FUssp0GwcmgGdFZA=; b=ojRn+ttfAoAzfwXu15IKscbKPZzRMR5e8B+Kx31EZvcKpEbvM36vZOvoI6JJQIfcN0 U5zZl4QNyiIBjO+t3dtlfRy20g+rxEcQUNUuGQHrmleAQ9zpiTA6NhwTUoVbXahO7mF6 0Aarht6CO0Gd2W/engNIBUzsoi6Jat6fg7czz8Q5rJ06vgRMTuWzZFVj/RHaZfdDKNhc +IzWZJdNqwi2vjJf/Z3ocg1Sw1KWyUo5aL6I/Eqo5gnkZoFEpV3ECAL2RiNpBjD9N/pa W6PPsL/yJbyPtw/A/VnlQkyIc58DSHLHaId6Kc3Nmu4h2xK32dGQzEQg1oa3B4ycQFlI gDgg== X-Forwarded-Encrypted: i=1; AJvYcCVPybEdOpK6bqdUIijY35zX+LvxCrlJg5yCBFpaeV3IGa2r40L17bHkETI+jfpXZpub0WMGqzVXjDjmNQ==@lists.postgresql.org X-Gm-Message-State: AOJu0YyV7rY33tM2H9C9vFIVauZuEPjugEPZuxEo2lzJfUCzkskCfaBB EhqKfkvhQQiKLql8j7LaxBIWJk0Ps1zM1UlPPk17UuhNL6JjcVfuuBQZAl/EWZJ5Ntoh6dhf0pH fXA5mq6KM7+AhnM7bDJSKRdcXxQ4= X-Google-Smtp-Source: AGHT+IFD7lN73TBudpOjhQz7MOeQtqyjB2Flt8GabPEyIgiE+V6wsyoVJU1I9ByZwhuGWxPRA8aidpMlobYUmeY+Ork= X-Received: by 2002:a05:6a20:2585:b0:1db:f519:1a7e with SMTP id adf61e73a8af0-1dc22b95820mr16973248637.42.1731307566216; Sun, 10 Nov 2024 22:46:06 -0800 (PST) MIME-Version: 1.0 References: <216310.1731300338@sss.pgh.pa.us> In-Reply-To: <216310.1731300338@sss.pgh.pa.us> From: Edwin UY Date: Mon, 11 Nov 2024 19:45:29 +1300 Message-ID: Subject: Re: How to check if session is a hung thread/session To: Tom Lane Cc: "David G. Johnston" , "pgsql-admin@lists.postgresql.org" Content-Type: multipart/alternative; boundary="0000000000007b3d4d06269d74f1" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --0000000000007b3d4d06269d74f1 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thanks Tom, OK, I've decided to 'painfully' look at the PostgreSQL RDS logs and it is showing something like below. There seems to be a locking/deadlock issue of some sort somewhere. I have checked the other days' log prior to the patching and these appear to be a 'normal' occurrence for this database and it wasn't affecting the application nonetheless. After the patching, it starts affecting the application. Not sure what else I can check on the Aurora PostgreSQL RDS end. I may request them to restart the app server. [25751]:LOG: process 25751 still waiting for ShareLock on transaction 114953443 after 1000.054 ms [25751]:DETAIL: Process holding the lock: 22297. Wait queue: 25751. [25751]:CONTEXT: while locking tuple (1,17) in relation "[table_name]" [25751]:STATEMENT: [SQL_STATEMENT] for update [25751]:LOG: process 25751 acquired ShareLock on transaction 114953443 after 4756.967 ms [25751]:CONTEXT: while locking tuple (1,17) in relation " [table_name] " On Mon, Nov 11, 2024 at 5:45=E2=80=AFPM Tom Lane wrote: > Edwin UY writes: > > I thought it could be the backend has sent something back to the client > but > > it never received it and it just kept on doing the same at some > intervals. > > Your pg_stat_activity output shows the backend is idle, meaning it's > waiting for a client command. While the session has been around for > days, we can see the last client command was about 47 minutes ago, > from your "now() - pg_stat_activity.query_start AS duration" column. > I see no reason to think there is anything interesting here at all, > except for a client that is sitting doing nothing for long periods. > > regards, tom lane > --0000000000007b3d4d06269d74f1 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thanks Tom,

OK, I've decided to 'painfully' look at the P= ostgreSQL RDS logs and it is showing something like below.
T= here seems to be a locking/deadlock issue of some sort somewhere.
I have checked the other days' log prior to the patching and thes= e appear to be a 'normal' occurrence for this database and it wasn&= #39;t affecting the application nonetheless.
After the patching, = it starts affecting the application. Not sure what else I can check on the = Aurora PostgreSQL RDS end. I may request them to restart the app server.

[25751]:LOG: =C2=A0process 25= 751 still waiting for ShareLock on transaction 114953443 after 1000.054 ms<= /span>
[25751]:DETAIL: =C2= =A0Process holding the lock: 22297. Wait queue: 25751.
[25= 751]:CONTEXT: =C2=A0while locking tuple (1,17) in relation "[table_nam= e]"
[25751]:STATEMENT:=C2=A0 [SQL_STATE= MENT] for update
[25751]:LOG: =C2=A0process 25751 acquired ShareLock on = transaction 114953443 after 4756.967 ms
[25751]:CONTEXT: =C2=A0while loc= king tuple (1,17) in relation "
[table_name] "


On Mo= n, Nov 11, 2024 at 5:45=E2=80=AFPM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Edwin UY <edwin.uy@gmail.com> writes: > I thought it could be the backend has sent something back to the clien= t but
> it never received it and it just kept on doing the same at some interv= als.

Your pg_stat_activity output shows the backend is idle, meaning it's waiting for a client command.=C2=A0 While the session has been around for days, we can see the last client command was about 47 minutes ago,
from your "now() - pg_stat_activity.query_start AS duration" colu= mn.
I see no reason to think there is anything interesting here at all,
except for a client that is sitting doing nothing for long periods.

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 regards, tom lane
--0000000000007b3d4d06269d74f1--