pg.ddx.io  pgsql-general@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: gzh <gzhcoder@126.com>
To: Laurenz Albe <laurenz.albe@cybertec.at>
Cc: pgsql-general@lists.postgresql.org
Subject: Re: An I/O error occured while sending to the backend
Date: Fri, 18 Nov 2022 11:25:07 +0800 (CST)
Message-ID: <160e6ef1.26e2.18488c4c53d.Coremail.gzhcoder@126.com> (raw)
In-Reply-To: <240a2e90ec4e940cd71e42d25435e2f3d4e6a4f9.camel@cybertec.at>
References: <449f5c48.3136.1847ed35c08.Coremail.gzhcoder@126.com>
	<240a2e90ec4e940cd71e42d25435e2f3d4e6a4f9.camel@cybertec.at>

Hi,  Rob


>Is this true for large numbers of lines in you csv?  Originally you said 
>something like 'adding 1000 lines' broke your app. Are you using 
>CopyManager?
Thank you for your reply. I didn't use CopyManager.
②'s query statement query the data inserted by ①, and a large amount of data inserted will cause performance problem.
I misunderstood, it's not ① that affects ②, it should be that ②'s query statement has a performance problem. 
The Encrypt function is used in ②'s query statement to encrypt the query result, and improper use of this function leads to poor performance. 
Sometimes 2~4 seconds to return the result, sometimes 7~19 seconds to return the result. 
We optimized the query statement to reduce the query time to less than 1 second and now the application works well.

















At 2022-11-16 14:57:41, "Laurenz Albe" <laurenz.albe@cybertec.at> wrote:
>On Wed, 2022-11-16 at 13:04 +0800, gzh wrote:
>> I have developed an application using the jdbc driver and 
>> connecting to a postgresql database .
>> The application mainly does the following two things:
>> ① read data from the CSV file and insert it into the database
>> ② perform a database query
>> In my application, the above ① and ② are executed asynchronously.
>> Everything was working fine until I increase the amount of CSV data to more than 1000.
>> Randomly they receive an error, the stack trace of which is below.
>>  
>> Caused by: java.net.SocketTimeoutException: Read timed out
>> 	at java.base/java.net.SocketInputStream.socketRead0(Native Method)
>> 	at java.base/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
>> 	at java.base/java.net.SocketInputStream.read(SocketInputStream.java:168)
>> 	at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
>> 	at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:478)
>> 	at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:472)
>> 	at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:70)
>> 	at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1454)
>> 	at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:1065)
>> 	at org.postgresql.core.VisibleBufferedInputStream.readMore(VisibleBufferedInputStream.java:161)
>
>Well, set a longer socket timeout if you need to run long SQL statements, or tune
>those statements to be faster.
>
>https://jdbc.postgresql.org/documentation/use/#connection-parameters has more
>information about "socketTimeout".
>
>Yours,
>Laurenz Albe


view thread (5+ messages)

Message-ID: <160e6ef1.26e2.18488c4c53d.Coremail.gzhcoder@126.com>
Permalink:  ../160e6ef1.26e2.18488c4c53d.Coremail.gzhcoder@126.com/
Also on:    postgresql.org/message-id/160e6ef1.26e2.18488c4c53d.Coremail.gzhcoder@126.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-general@postgresql.org
  Cc: gzhcoder@126.com, laurenz.albe@cybertec.at
  Subject: Re: An I/O error occured while sending to the backend
  In-Reply-To: <160e6ef1.26e2.18488c4c53d.Coremail.gzhcoder@126.com>
  References: <449f5c48.3136.1847ed35c08.Coremail.gzhcoder@126.com> <240a2e90ec4e940cd71e42d25435e2f3d4e6a4f9.camel@cybertec.at> <160e6ef1.26e2.18488c4c53d.Coremail.gzhcoder@126.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox