Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1tDiJ8-002hgl-39 for pgsql-general@arkaria.postgresql.org; Wed, 20 Nov 2024 10:51:38 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1tDiJ6-004cqT-Av for pgsql-general@arkaria.postgresql.org; Wed, 20 Nov 2024 10:51:36 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1tDiJ5-004cqH-UA for pgsql-general@lists.postgresql.org; Wed, 20 Nov 2024 10:51:35 +0000 Received: from mail-oo1-xc2a.google.com ([2607:f8b0:4864:20::c2a]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1tDiJ2-002rg4-GB for pgsql-general@lists.postgresql.org; Wed, 20 Nov 2024 10:51:34 +0000 Received: by mail-oo1-xc2a.google.com with SMTP id 006d021491bc7-5ebc1af9137so914929eaf.2 for ; Wed, 20 Nov 2024 02:51:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lifetrenz.com; s=google; t=1732099891; x=1732704691; darn=lists.postgresql.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=ZbTFlS7f63ykCxilGsWsAxw7hkOhaLr6rPTg9zec/VA=; b=DmJ1hJvx0b5cMveOt4Qtwa32rKMnVSc9WjD5bhvfC0KlOo0XFSOLzVEiQI2EYon3Hb 9pE5EXLffkzWRxioHTBWwDyQTFOIl0HrOhJuVsaeOW16tT8ZB/mPlmZm+V9ktGnYNeyR xVaDP6gGgPHBsF0LLacUR3fOJH/GHm6DXe6XR+jNIuv1kT9z9i/+uZSnBP0T953YLcGs QrrDGvLK8amb3FbqwAUMjgbg62zryCEJODDMCCFecpraLhMM/jiwtXVRoD2If2LFeF0G ncwCtIlaxb8+lb8BlTEqBDfS6x13+88EZ6MaRGec3SeXzGUFdmcXahpoVG9VftK0YIzg 6zSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732099891; x=1732704691; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=ZbTFlS7f63ykCxilGsWsAxw7hkOhaLr6rPTg9zec/VA=; b=aNemuWQgUrARqwGssE+kgP+dExZvhAJywhp2wMmJaD98CbX9AL5EOlqZvOyuVwdtsL UjdbIBbAQQXmjspJ20dJ6DcJqGOpjQXQtYp1YFHTvk5kT+TghSQfNV03rfsVruc1Z/7k /lCIsBAAmyCteEICyJeV+65ZT48PDc3K8+FnLShR2cis2QupJx70Dz2zR6IIQ+KKHg5A GVgQoygdyWJyi/vyVRwY5V0Ot9K1ZYa2wECAnIYNN/B+0ylatNVEwd0rYyqUPMTyxiLy mD4pSJIpK0cc/zGEyTtV5EScGLWbHhOhhbSEfWPovmzrAyL1QLx0AJn2or2w9C8fTuR/ kA3A== X-Gm-Message-State: AOJu0YzMIDbyQuknUG9T4aWGKDgLOnHXAtAmuaEWHfVPpz4VIoMrfdGs BcSh9aWua7/Md4+RcFrw8Pl5P8Q8GYgdA4pM38IJPMH8I4iNjvvev5muBW1q+PA0IfYzKau0woI owlBadFVmAUv75o9Px6BQRCczsJy+kdL0kcWGUnn/WVHwTyFh3mZxWaFlvbm2fMO7/feEVJr8PE EywsQuR7kCk1hwMiqgRocXXbcw9qrajF3uhdVCgs78pkHUJd7c X-Google-Smtp-Source: AGHT+IGm5MISZYKSAI89XiSgi6el4QsG7JmZjphDjWFONvd6IjxvOnIcnephEY+JX2aJpwecVIPrSp48KEEl3/GJ+aM= X-Received: by 2002:a4a:edcc:0:b0:5e5:c456:8996 with SMTP id 006d021491bc7-5eee8282755mr1952348eaf.4.1732099891125; Wed, 20 Nov 2024 02:51:31 -0800 (PST) MIME-Version: 1.0 From: Sreejith P Date: Wed, 20 Nov 2024 16:20:55 +0530 Message-ID: Subject: Suddenly all queries moved to seq scan To: pgsql-general@lists.postgresql.org Content-Type: multipart/alternative; boundary="000000000000b9f8b2062755ee69" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --000000000000b9f8b2062755ee69 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi, We are using PostgresQL 10 in our production database. We have around 890 req /s request on peak time. We have 1 primary and 4 slave databases as well in the same postgres cluster. 2 days back we applied some patches in the primary server and restarted. We didn't do anything on the secondary server. Next day, After 18 hours all our queries from secondary servers started taking too much time. queries were working in 2 sec started taking 80 seconds. Almost all queries behaved the same way. After half an hour of outage we restarted all db servers and system back to normal. Still we are not able to understand the root case. We couldn't find any error log or fatal errors. During the incident, in one of the read server disks was full. We couldn't see any replication lag or query cancellation due to replication. please help Regards Sreejith --=20 =C2=A0 *Solutions for Care Anywhere* *dWise HealthCare IT Solutions Pvt.=20 Ltd.* | www.lifetrenz.com *Disclaimer*: The=20 information and attachments contained in this email are intended=20 for=20 exclusive use of the addressee(s) and may contain confidential or=20 privileged information. If you are not the intended recipient, please=20 notify the sender immediately and destroy all copies of this message and =20 any attachments. The views expressed in this email are, unless=20 otherwise=20 stated, those of the author and not those of dWise HealthCare IT Solutions= =20 or its management. --000000000000b9f8b2062755ee69 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi,

We are using Postg= resQL=C2=A010 in our production database.=C2=A0 We have around 890 req /s r= equest on peak time.

We have 1 primary and 4 slave= databases as well in the same postgres cluster.=C2=A0

=
2 days back we applied some patches in the primary server and restarte= d. We didn't do anything on the secondary server.

<= div>Next day, After 18 hours all our queries from secondary servers started= taking too much time.=C2=A0 queries were working in 2 sec started taking 8= 0 seconds. Almost all queries behaved the same way.

After half an hour of outage we restarted all db servers and system back = to normal.

Still we are not able to understand the= root case. We couldn't=C2=A0find any error log or fatal errors.=C2=A0 = During the incident, in=C2=A0 one of the read server disks was full. We cou= ldn't=C2=A0see any replication lag or query cancellation=C2=A0due to re= plication.

please help

Re= gards
Sreejith



=


=C2=A0

Solutions for= Care Anywhere


dWise HealthCare IT Solutions Pvt. Ltd. | www.l= ifetrenz.com
Discla= imer: The information and attachments contained in this email are intended=20 for exclusive use of the addressee(s) and may contain confidential or=20 privileged information. If you are not the intended recipient, please=20 notify the sender immediately and destroy all copies of this message and any attachments. The views expressed in this email are, unless=20 otherwise stated, those of the author and not those of dWise HealthCare IT = Solutions or its management.
--000000000000b9f8b2062755ee69--