Feedback-ID: i76984098:Fastmail
Message-ID: <4cfe720e-4283-43fc-8d72-8fa11e2f741a@aklaver.com>
Date: Fri, 20 Sep 2024 14:11:38 -0700
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: IO related waits
To: veem v <veema0000@gmail.com>
Cc: Greg Sabino Mullane <htamfids@gmail.com>,
 Christophe Pettus <xof@thebuild.com>,
 pgsql-general <pgsql-general@lists.postgresql.org>
References: <CAB+=1TWZNvMhVthJ2iKs_Q4qBzMw-v_oaSz7HbFE_P_qC5jMFA@mail.gmail.com>
 <3dddea5e-52ab-4075-970d-a87b0c921ae7@aklaver.com>
 <CAB+=1TWdRd2sBw7-vXCovH_VHLANh+aSaU-WyJ2m8tL4TkF=8g@mail.gmail.com>
 <225d1bc1-5117-4c72-85a1-bac6355fb659@aklaver.com>
 <CAB+=1TV-b8=RGKVFN0qj5Nxdgbj-JYCNh4ZqFjxxmQPpFD6=PQ@mail.gmail.com>
 <fd6afa3b-f5db-4632-8e25-678ef66703d5@aklaver.com>
 <CAB+=1TXn-TZFC5QAFDshpRpnNhGrZ847sdaivNJRgot-6BuerA@mail.gmail.com>
 <CAB+=1TWhGgybtCdU7oR3oghhXG6vULUEkebG1DbT+bc1v7yp4w@mail.gmail.com>
 <586d988d-978b-4b08-84b4-ef3ebb8736a8@aklaver.com>
 <CAB+=1TXcLJsZoGZ2qzv5ehauOa3GX_-cHth4NvZ2mgFsJPCffA@mail.gmail.com>
 <CAKAnmmJhsNL=f+z7R018L6NrLQuq_7+quqZkGUKbzHRctHaY3w@mail.gmail.com>
 <bc8363b5-28d0-4218-b3d6-73c7d78717da@aklaver.com>
 <CAB+=1TXEJTMtypsZrNhGVEZxRV6jBop4hmMaR5DDsTLOoSqsdQ@mail.gmail.com>
Content-Language: en-US
From: Adrian Klaver <adrian.klaver@aklaver.com>
In-Reply-To: <CAB+=1TXEJTMtypsZrNhGVEZxRV6jBop4hmMaR5DDsTLOoSqsdQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Archived-At: <https://www.postgresql.org/message-id/4cfe720e-4283-43fc-8d72-8fa11e2f741a%40aklaver.com>
Precedence: bulk


On 9/20/24 1:01 PM, veem v wrote:
> 
> On Thu, 19 Sept, 2024, 8:40 pm Adrian Klaver, <adrian.klaver@aklaver.com 
> <mailto:adrian.klaver@aklaver.com>> wrote:
> 
>     On 9/19/24 05:24, Greg Sabino Mullane wrote:
>      > On Thu, Sep 19, 2024 at 5:17 AM veem v <veema0000@gmail.com
>     <mailto:veema0000@gmail.com>
> 
>      > This is really difficult to diagnose from afar with only snippets of
>      > logs and half-complete descriptions of your business logic. Pull
>      > everyone involved into a room with a whiteboard, and produce a
>     document
>      > describing exactly what your application does, and how it is
>     doing it.
>      > Switch from reactive to proactive.
> 
> 
> Able to reproduce this deadlock graph as below.  Now my question is , 
> this is a legitimate scenario in which the same ID can get inserted from 
> multiple sessions and in such cases it's expected to skip that (thus "On 
> conflict Do nothing" is used) row. But as we see it's breaking the code

Yeah, as I see it that would not work with concurrent uncommitted 
sessions as it would be unresolved whether a conflict actually exists 
until at least one of the sessions completes.

> with deadlock error during race conditions where a lot of parallel 
> threads are operating. So how should we handle this scenario? Will 
> setting the "lock_timeout" parameter at session level will help us 
> anyway here?

Serializable transaction?:

https://www.postgresql.org/docs/current/transaction-iso.html#XACT-SERIALIZABLE

Or change the application code to not have this:

"... legitimate scenario in which the same ID can get inserted from 
multiple sessions ..."
-- 
Adrian Klaver
adrian.klaver@aklaver.com