public inbox for [email protected]  

help / color / mirror / Atom feed

From: Sreejith P <[email protected]>
To: Daniel Gustafsson <[email protected]>
Cc: [email protected]
Subject: Re: Suddenly all queries moved to seq scan
Date: Wed, 20 Nov 2024 19:38:49 +0530
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <CAOW_W8PZ7Op_vOT9szMnpZE4jOmP09NWfc3M1ORTFMzjebpK5w@mail.gmail.com>
	<[email protected]>



> On 20 Nov 2024, at 6:32 PM, Daniel Gustafsson <[email protected]> wrote:
> 
>> On 20 Nov 2024, at 11:50, Sreejith P <[email protected]> wrote:
> 
>> We are using PostgresQL 10 in our production database.  We have around 890 req /s request on peak time.
> 
> PostgreSQL 10 is well out of support and does not receive bugfixes or security
> fixes, you should plan a migration to a supported version sooner rather than
> later.
> 
>> 2 days back we applied some patches in the primary server and restarted. We didn't do anything on the secondary server.
> 
> Patches to the operating system, postgres, another application? 
PostgreSQL Common 10.23-6 
> 
>> Next day, After 18 hours all our queries from secondary servers started taking too much time.  queries were working in 2 sec started taking 80 seconds. Almost all queries behaved the same way.
>> 
>> After half an hour of outage we restarted all db servers and system back to normal.
>> 
>> Still we are not able to understand the root case. We couldn't find any error log or fatal errors.  During the incident, in  one of the read server disks was full. We couldn't see any replication lag or query cancellation due to replication.
> 
> You say that all queries started doing sequential scans, is that an assumption
> from queries being slow or did you capture plans for the queries which be
> compared against "normal" production plans?.

Queries were taking 20 ms started taking 60 seconds. So have done SQL analyse to understand about query plan. There we found that query planner taking seq scan instead in index scan.

I would like to add one ore point.  A delete query were running in DB from 2 days for deleting around 80 million records. 
> 
> --
> Daniel Gustafsson
> 


-- 




 

*Solutions for Care Anywhere*
*dWise HealthCare IT Solutions Pvt. 
Ltd.* | www.lifetrenz.com <http://www.lifetrenz.com;
*Disclaimer*:
 The 
information and attachments contained in this email are intended 
for 
exclusive use of the addressee(s) and may contain confidential or 
privileged information. If you are not the intended recipient, please 
notify the sender immediately and destroy all copies of this message and
 
any attachments. The views expressed in this email are, unless 
otherwise 
stated, those of the author and not those of dWise HealthCare IT Solutions 
or its management.

view thread (4+ messages)

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected]
  Subject: Re: Suddenly all queries moved to seq scan
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox