pg.ddx.io pgsql-bugs@postgresql.org mailing list archive
help / color / mirror / Atom feedBUG #18569: Memory leak in Postgres Enterprise server
13+ messages / 6 participants
[nested] [flat]
* BUG #18569: Memory leak in Postgres Enterprise server
@ 2024-08-06 08:34 PG Bug reporting form <noreply@postgresql.org>
2024-08-06 11:15 ` Re: BUG #18569: Memory leak in Postgres Enterprise server Daniel Gustafsson <daniel@yesql.se>
0 siblings, 1 reply; 13+ messages in thread
From: PG Bug reporting form @ 2024-08-06 08:34 UTC (permalink / raw)
To: pgsql-bugs@lists.postgresql.org; +Cc: absahu@informatica.com
The following bug has been logged on the website:
Bug reference: 18569
Logged by: Abhisek Sahu
Email address: absahu@informatica.com
PostgreSQL version: 16.3
Operating system: Redhat Linux8
Description:
We are encountering a issue with our PostgreSQL database, which is impacting
our operations.
Issue Description:
Our Informatica repository setup is enabled with PostgreSQL 16.2 database.
We are running multiple concurrent operations for extended periods (10 to 12
hours), which is resulting in database connection errors.
Error Details:
We have observed that memory consumption by PostgreSQL processes increases
gradually over time. Eventually, this leads to out-of-memory errors and
process termination. The log entries are as follows-
We also noticed that until the application either hangs or we manually
terminate the connection on our side, the memory used by these processes on
the PostgreSQL server continues to increase. This has led to severe
performance issues and frequent database connection errors.
Additional Information:
PostgreSQL Server/Client Version: PostgreSQL 16.2
Connector in Use: LibPQ
We have also attached our Repository logs, PostgreSQL server logs and DB
logs. Based on our analysis, it seems that processes are being killed due to
out-of-memory errors, as indicated by the logs from around 17:02
We would appreciate your assistance in analyzing and resolving this memory
leak issue as soon as possible.
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: BUG #18569: Memory leak in Postgres Enterprise server
2024-08-06 08:34 BUG #18569: Memory leak in Postgres Enterprise server PG Bug reporting form <noreply@postgresql.org>
@ 2024-08-06 11:15 ` Daniel Gustafsson <daniel@yesql.se>
2024-08-06 12:01 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Sahu, Abhisek Kumar <absahu@informatica.com>
0 siblings, 1 reply; 13+ messages in thread
From: Daniel Gustafsson @ 2024-08-06 11:15 UTC (permalink / raw)
To: absahu@informatica.com; pgsql-bugs@lists.postgresql.org
> On 6 Aug 2024, at 10:34, PG Bug reporting form <noreply@postgresql.org> wrote:
> PostgreSQL Server/Client Version: PostgreSQL 16.2
Your report mentions "Postgres Enterprise Server", are you running a product
from a vendor like EDB or are you running postgres installed by yourself via a
package manager (or similar)? If you are buying postgres from a vendor then
please contact them, if not you are in the right place.
Upgrading to the latest 16.3 is also recommended even though there are no
references to memory leak fixes in the release notes.
> We have also attached our Repository logs, PostgreSQL server logs and DB
> logs.
No, there are no logs attached (I don't think the bugreporting form support
attaching any files). Please provide additional information as there is
nothing to go in the report.
--
Daniel Gustafsson
^ permalink raw reply [nested|flat] 13+ messages in thread
* RE: BUG #18569: Memory leak in Postgres Enterprise server
2024-08-06 08:34 BUG #18569: Memory leak in Postgres Enterprise server PG Bug reporting form <noreply@postgresql.org>
2024-08-06 11:15 ` Re: BUG #18569: Memory leak in Postgres Enterprise server Daniel Gustafsson <daniel@yesql.se>
@ 2024-08-06 12:01 ` Sahu, Abhisek Kumar <absahu@informatica.com>
2024-08-07 05:35 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Mathias, Renci <rmathias@informatica.com>
0 siblings, 1 reply; 13+ messages in thread
From: Sahu, Abhisek Kumar @ 2024-08-06 12:01 UTC (permalink / raw)
To: Daniel Gustafsson <daniel@yesql.se>; pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>; Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>; Sachan, Vishal <vsachan@informatica.com>; +Cc: Mathias, Renci <rmathias@informatica.com>; Kumar, Gaurav <gaurkumar@informatica.com>
Hi Daniel,
Thank you for your response.
@Sachan, Vishal please attach all the logs, including the ODL tracing.
Adding @Yeddula, Madhusudhan reddy [CONTINGENT WORKER] DBA to answer the below queries.
Regards
Abhisek
-----Original Message-----
From: Daniel Gustafsson <daniel@yesql.se>
Sent: Tuesday, August 6, 2024 4:45 PM
To: Sahu, Abhisek Kumar <absahu@informatica.com>; pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #18569: Memory leak in Postgres Enterprise server
Caution: This email originated from outside of the organization. Review for Phishing!
> On 6 Aug 2024, at 10:34, PG Bug reporting form <noreply@postgresql.org> wrote:
> PostgreSQL Server/Client Version: PostgreSQL 16.2
Your report mentions "Postgres Enterprise Server", are you running a product from a vendor like EDB or are you running postgres installed by yourself via a package manager (or similar)? If you are buying postgres from a vendor then please contact them, if not you are in the right place.
Upgrading to the latest 16.3 is also recommended even though there are no references to memory leak fixes in the release notes.
> We have also attached our Repository logs, PostgreSQL server logs and
> DB logs.
No, there are no logs attached (I don't think the bugreporting form support attaching any files). Please provide additional information as there is nothing to go in the report.
--
Daniel Gustafsson
^ permalink raw reply [nested|flat] 13+ messages in thread
* RE: BUG #18569: Memory leak in Postgres Enterprise server
2024-08-06 08:34 BUG #18569: Memory leak in Postgres Enterprise server PG Bug reporting form <noreply@postgresql.org>
2024-08-06 11:15 ` Re: BUG #18569: Memory leak in Postgres Enterprise server Daniel Gustafsson <daniel@yesql.se>
2024-08-06 12:01 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Sahu, Abhisek Kumar <absahu@informatica.com>
@ 2024-08-07 05:35 ` Mathias, Renci <rmathias@informatica.com>
2024-08-07 11:14 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>
0 siblings, 1 reply; 13+ messages in thread
From: Mathias, Renci @ 2024-08-07 05:35 UTC (permalink / raw)
To: Daniel Gustafsson <daniel@yesql.se>; pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>; Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>; Sahu, Abhisek Kumar <absahu@informatica.com>; Sachan, Vishal <vsachan@informatica.com>; +Cc: Kumar, Gaurav <gaurkumar@informatica.com>
Hi @Daniel,
The log files are attached for your reference.
Please let us know if you have any queries.
Thanks,
Renci Monthi Mathias | QA Engineer
Bagmane Tech Park, C V Raman Nagar, Bangalore - 560093
Email: rmathias@informatica.com
-----Original Message-----
From: Sahu, Abhisek Kumar <absahu@informatica.com>
Sent: Tuesday, August 6, 2024 5:31 PM
To: Daniel Gustafsson <daniel@yesql.se>; pgsql-bugs@lists.postgresql.org; Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>; Sachan, Vishal <vsachan@informatica.com>
Cc: Mathias, Renci <rmathias@informatica.com>; Kumar, Gaurav <gaurkumar@informatica.com>
Subject: RE: BUG #18569: Memory leak in Postgres Enterprise server
Hi Daniel,
Thank you for your response.
@Sachan, Vishal please attach all the logs, including the ODL tracing.
Adding @Yeddula, Madhusudhan reddy [CONTINGENT WORKER] DBA to answer the below queries.
Regards
Abhisek
-----Original Message-----
From: Daniel Gustafsson <daniel@yesql.se>
Sent: Tuesday, August 6, 2024 4:45 PM
To: Sahu, Abhisek Kumar <absahu@informatica.com>; pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #18569: Memory leak in Postgres Enterprise server
Caution: This email originated from outside of the organization. Review for Phishing!
> On 6 Aug 2024, at 10:34, PG Bug reporting form <noreply@postgresql.org> wrote:
> PostgreSQL Server/Client Version: PostgreSQL 16.2
Your report mentions "Postgres Enterprise Server", are you running a product from a vendor like EDB or are you running postgres installed by yourself via a package manager (or similar)? If you are buying postgres from a vendor then please contact them, if not you are in the right place.
Upgrading to the latest 16.3 is also recommended even though there are no references to memory leak fixes in the release notes.
> We have also attached our Repository logs, PostgreSQL server logs and
> DB logs.
No, there are no logs attached (I don't think the bugreporting form support attaching any files). Please provide additional information as there is nothing to go in the report.
--
Daniel Gustafsson
Attachments:
PCRS_logs.txt (text/plain, 7.1K) download
2024-07-29 17:02:02.230 IST : ERROR : (13828 | 14312) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : OBJM_54543 : DataBase error:
PostgreSQL database returns status [PGRES_FATAL_ERROR], message [server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
]
Error in preparing statement [SELECT NAME_SPACE, NAME, ID, STATUS, PRIVILEGES1, PRIVILEGES2, PRIVILEGES3, PRIVILEGES4, PRIVILEGES5 from OPB_USER_GROUP WHERE TYPE = 1 AND STATUS < $1 AND NAME = $2 AND NAME_SPACE = $3]
Database driver error...
Function Name : Prepare
SQL Stmt : SELECT NAME_SPACE, NAME, ID, STATUS, PRIVILEGES1, PRIVILEGES2, PRIVILEGES3, PRIVILEGES4, PRIVILEGES5 from OPB_USER_GROUP WHERE TYPE = 1 AND STATUS < ? AND NAME = ? AND NAME_SPACE = ?
PostgreSQL database returns status [PGRES_FATAL_ERROR], message [server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
]
Error in preparing statement [SELECT NAME_SPACE, NAME, ID, STATUS, PRIVILEGES1, PRIVILEGES2, PRIVILEGES3, PRIVILEGES4, PRIVILEGES5 from OPB_USER_GROUP WHERE TYPE = 1 AND STATUS < $1 AND NAME = $2 AND NAME_SPACE = $3].
2024-07-29 17:02:01.190 IST : ERROR : (13828 | 16912) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : CNX_53140 : Internal error. Exiting worker thread due to a broken connection.
2024-07-29 17:02:01.186 IST : INFO : (13828 | 16912) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : CNX_53124 : High availability is turned on for this connection. The max resilience timeout is set to 180. The heartbeat interval is set to 60.
2024-07-29 17:02:01.184 IST : INFO : (13828 | 16912) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : CNX_53032 : User Administrator using application pmrep on host invw22pcqa05.informatica.com (10.65.42.126), port 60893, logged in successfully.
2024-07-29 17:02:01.019 IST : INFO : (13828 | 14804) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : CNX_53144 : Client exiting normally.
2024-07-29 17:02:01.016 IST : INFO : (13828 | 14804) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : CNX_53035 : Received logout request from user test_user3 on host invw22pcqa05.informatica.com (10.65.42.126) port 59504.
2024-07-29 17:02:00.921 IST : ERROR : (13828 | 13024) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : CNX_53145 : Unable to process request, opcode=120, error=-2.
2024-07-29 17:02:00.917 IST : ERROR : (13828 | 13024) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : CNX_53022 : Failed to send reply.
2024-07-29 17:02:00.914 IST : ERROR : (13828 | 13024) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : REP_51046 : Internal error : Cannot send object of type RReplyMsg.
2024-07-29 17:01:13.126 IST : INFO : (13828 | 13144) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : CNX_53116 : Broken connection detected for cnx id: 1241. RA will wait for a reconnection.
2024-07-29 17:01:07.851 IST : ERROR : (13828 | 16912) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : CNX_53111 : Error accessing the repository database. Repository Agent retry attempt to connect
2024-07-29 17:01:07.847 IST : ERROR : (13828 | 16912) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : OBJM_54538 : Unable to connect to the repository Stress_Repository on database PSQL16.
2024-07-29 17:01:07.845 IST : ERROR : (13828 | 16912) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : REP_51002 : Database connection timeout. Unable to create a database connection. Please check your database server.
2024-07-29 17:00:42.120 IST : INFO : (13828 | 13144) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : CNX_53115 : Missing heartbeat from cnx id: 1241.
2024-07-29 16:59:40.112 IST : INFO : (13828 | 13144) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : CNX_53115 : Missing heartbeat from cnx id: 1241.
2024-07-29 16:59:07.230 IST : ERROR : (13828 | 14812) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : REP_51002 : Failed to logon to Database server
Database driver error...
Function Name : Connect
connection to server at "10.65.139.230", port 5432 failed: server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
Database Error: Failed to connect to database using user [nfs_102] and connection string [PSQL16].
2024-07-29 16:58:07.834 IST : INFO : (13828 | 16912) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : CNX_53130 : Client repository version is 189, data version is 98, and product version is 10.5.6.
2024-07-29 16:58:07.831 IST : INFO : (13828 | 16912) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : CNX_53039 : Received TCP connection from host invw22pcqa05.informatica.com (10.65.42.126), port 60893. Connection ID: 1241.
2024-07-29 16:58:07.102 IST : INFO : (13828 | 13144) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : CNX_53123 : Connection terminated (for connection id 1238) since it has remained broken beyond resilience time.
2024-07-29 16:56:34.092 IST : INFO : (13828 | 13144) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : CNX_53123 : Connection terminated (for connection id 1240) since it has remained broken beyond resilience time.
2024-07-29 16:54:30.081 IST : INFO : (13828 | 13144) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : CNX_53116 : Broken connection detected for cnx id: 1238. RA will wait for a reconnection.
2024-07-29 16:54:30.079 IST : INFO : (13828 | 13144) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : CNX_53115 : Missing heartbeat from cnx id: 1238.
2024-07-29 16:53:28.071 IST : INFO : (13828 | 13144) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : CNX_53115 : Missing heartbeat from cnx id: 1238.
2024-07-29 16:52:57.066 IST : INFO : (13828 | 13144) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : CNX_53116 : Broken connection detected for cnx id: 1240. RA will wait for a reconnection.
2024-07-29 16:52:26.063 IST : INFO : (13828 | 13144) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : CNX_53115 : Missing heartbeat from cnx id: 1240.
2024-07-29 16:52:26.060 IST : INFO : (13828 | 13144) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : CNX_53115 : Missing heartbeat from cnx id: 1238.
2024-07-29 16:51:24.051 IST : INFO : (13828 | 13144) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : CNX_53115 : Missing heartbeat from cnx id: 1240.
2024-07-29 16:50:22.043 IST : ERROR : (13828 | 13144) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : CNX_53101 : Database error: Database connection timeout. Unable to create a database connection. Please check your database server.
2024-07-29 16:49:51.767 IST : INFO : (13828 | 14312) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : CNX_53130 : Client repository version is 189, data version is 98, and product version is 10.5.6.
2024-07-29 16:49:51.764 IST : INFO : (13828 | 14312) : (RS | Stress_Repository) : n_Windows.64_invw22pcqa05 : CNX_53039 : Received TCP connection from host invw22pcqa05.informatica.com (10.65.42.126), port 59509. Connection ID: 1240.
Postgres16.2_DB_logs.txt (text/plain, 21.7K) download
2024-07-29 17:02:01.997 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73201_C CURSOR FOR SELECT COUNT(*) FROM OPB_PICDSC , OPB_SRC_FLD, OPB_SRC WHERE OPB_PICDSC.PICID = OPB_SRC_FLD.PICID AND OPB_SRC_FLD.SRC_ID = OPB_SRC.SRC_ID AND OPB_SRC.VERSION_NUMBER = OPB_SRC_FLD.VERSION_NUMBER AND OPB_SRC.VERSION_NUMBER = OPB_PICDSC.VERSION_NUMBER AND OPB_SRC.IS_VISIBLE <> 2 AND OPB_SRC.SUBJ_ID = 178
2024-07-29 17:02:02.004 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73201_C
2024-07-29 17:02:02.042 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73201_C
2024-07-29 17:02:02.047 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73202_C CURSOR FOR SELECT COUNT(*) FROM OPB_WIDG_FLD_ATTR , OPB_SRC WHERE OPB_WIDG_FLD_ATTR.SESSION_ID = 0 AND OPB_WIDG_FLD_ATTR.MAPPING_ID = 0 AND OPB_WIDG_FLD_ATTR.OBJECT_TYPE = 1 AND OPB_WIDG_FLD_ATTR.OBJECT_ID = OPB_SRC.SRC_ID AND OPB_SRC.IS_VISIBLE <> 2 AND OPB_SRC.VERSION_NUMBER = OPB_WIDG_FLD_ATTR.VERSION_NUMBER AND OPB_SRC.SUBJ_ID = 178
2024-07-29 17:02:02.048 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73202_C
2024-07-29 17:02:02.090 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73202_C
2024-07-29 17:02:02.092 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73203_C CURSOR FOR SELECT COUNT(*) FROM OPB_WIDGET_ATTR , OPB_SRC WHERE OPB_WIDGET_ATTR.MAPPING_ID = 0 AND OPB_WIDGET_ATTR.WIDGET_ID = OPB_SRC.SRC_ID AND OPB_WIDGET_ATTR.WIDGET_TYPE = 1 AND OPB_SRC.IS_VISIBLE <> 2 AND OPB_SRC.VERSION_NUMBER = OPB_WIDGET_ATTR.VERSION_NUMBER AND OPB_SRC.SUBJ_ID = 178
2024-07-29 17:02:02.093 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73203_C
2024-07-29 17:02:02.099 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73203_C
2024-07-29 17:02:02.101 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73204_C CURSOR FOR SELECT COUNT(*) FROM OPB_METAEXT_VAL , OPB_SRC WHERE OPB_METAEXT_VAL.OBJECT_TYPE = 1 AND OPB_METAEXT_VAL.OBJECT_ID = OPB_SRC.SRC_ID AND OPB_SRC.IS_VISIBLE <> 2 AND OPB_SRC.VERSION_NUMBER = OPB_METAEXT_VAL.VERSION_NUMBER AND OPB_METAEXT_VAL.SUBJECT_ID = OPB_SRC.SUBJ_ID AND OPB_METAEXT_VAL.SUBJECT_ID = 178
2024-07-29 17:02:02.102 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73204_C
2024-07-29 17:02:02.110 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73204_C
2024-07-29 17:02:02.113 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73205_C CURSOR FOR SELECT COUNT(*) FROM OPB_XML_INFO , OPB_SRC WHERE OPB_XML_INFO.OBJECT_TYPE = 1 AND OPB_XML_INFO.OBJECT_ID = OPB_SRC.SRC_ID AND OPB_SRC.IS_VISIBLE <> 2 AND OPB_SRC.VERSION_NUMBER = OPB_XML_INFO.VERSION_NUMBER AND OPB_XML_INFO.SUBJECT_ID = OPB_SRC.SUBJ_ID AND OPB_XML_INFO.SUBJECT_ID = 178
2024-07-29 17:02:02.114 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73205_C
2024-07-29 17:02:02.118 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73205_C
2024-07-29 17:02:02.119 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73206_C CURSOR FOR SELECT COUNT(*) FROM OPB_XML_QUAL , OPB_SRC WHERE OPB_XML_QUAL.OBJECT_TYPE = 1 AND OPB_XML_QUAL.OBJECT_ID = OPB_SRC.SRC_ID AND OPB_SRC.IS_VISIBLE <> 2 AND OPB_SRC.VERSION_NUMBER = OPB_XML_QUAL.VERSION_NUMBER AND OPB_XML_QUAL.SUBJECT_ID = OPB_SRC.SUBJ_ID AND OPB_XML_QUAL.SUBJECT_ID = 178
2024-07-29 17:02:02.121 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73206_C
2024-07-29 17:02:02.122 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73206_C
2024-07-29 17:02:02.125 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73207_C CURSOR FOR SELECT COUNT(*) FROM OPB_TABLE_GROUP , OPB_SRC WHERE OPB_TABLE_GROUP.OBJECT_ID = OPB_SRC.SRC_ID AND OPB_TABLE_GROUP.OBJECT_TYPE = 1 AND OPB_SRC.IS_VISIBLE <> 2 AND OPB_SRC.VERSION_NUMBER = OPB_TABLE_GROUP.VERSION_NUMBER AND OPB_TABLE_GROUP.SUBJECT_ID = OPB_SRC.SUBJ_ID AND OPB_SRC.SUBJ_ID = 178
2024-07-29 17:02:02.126 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73207_C
2024-07-29 17:02:02.134 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73207_C
2024-07-29 17:02:02.137 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73208_C CURSOR FOR SELECT COUNT(*) FROM OPB_SEBL_MVL_LREL , OPB_SRC WHERE OPB_SRC.SRC_ID = OPB_SEBL_MVL_LREL.SRC_ID AND OPB_SRC.IS_VISIBLE <> 2 AND OPB_SRC.VERSION_NUMBER = OPB_SEBL_MVL_LREL.VERSION_NUMBER AND OPB_SRC.SUBJ_ID = OPB_SEBL_MVL_LREL.SUBJECT_ID AND OPB_SRC.SUBJ_ID = 178
2024-07-29 17:02:02.138 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73208_C
2024-07-29 17:02:02.139 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73208_C
2024-07-29 17:02:02.139 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73209_C CURSOR FOR SELECT COUNT(*) FROM OPB_SEBL_MVL_ATTR , OPB_SRC WHERE OPB_SRC.SRC_ID = OPB_SEBL_MVL_ATTR.SRC_ID AND OPB_SRC.IS_VISIBLE <> 2 AND OPB_SRC.VERSION_NUMBER = OPB_SEBL_MVL_ATTR.VERSION_NUMBER AND OPB_SRC.SUBJ_ID = OPB_SEBL_MVL_ATTR.SUBJECT_ID AND OPB_SRC.SUBJ_ID = 178
2024-07-29 17:02:02.140 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73209_C
2024-07-29 17:02:02.141 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73209_C
2024-07-29 17:02:02.142 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73210_C CURSOR FOR SELECT COUNT(*) FROM OPB_SEBL_JOIN_ATTR , OPB_SRC WHERE OPB_SRC.SRC_ID = OPB_SEBL_JOIN_ATTR.SRC_ID AND OPB_SRC.IS_VISIBLE <> 2 AND OPB_SRC.VERSION_NUMBER = OPB_SEBL_JOIN_ATTR.VERSION_NUMBER AND OPB_SRC.SUBJ_ID = OPB_SEBL_JOIN_ATTR.SUBJECT_ID AND OPB_SRC.SUBJ_ID = 178
2024-07-29 17:02:02.142 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73210_C
2024-07-29 17:02:02.143 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73210_C
2024-07-29 17:02:02.145 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73211_C CURSOR FOR SELECT COUNT(*) FROM OPB_SEBL_LINK , OPB_SRC WHERE OPB_SRC.SRC_ID = OPB_SEBL_LINK.SRC_ID AND OPB_SRC.IS_VISIBLE <> 2 AND OPB_SRC.VERSION_NUMBER = OPB_SEBL_LINK.VERSION_NUMBER AND OPB_SRC.SUBJ_ID = OPB_SEBL_LINK.SUBJECT_ID AND OPB_SRC.SUBJ_ID = 178
2024-07-29 17:02:02.146 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73211_C
2024-07-29 17:02:02.146 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73211_C
2024-07-29 17:02:02.148 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73212_C CURSOR FOR SELECT COUNT(*) FROM OPB_SEBL_LINKX , OPB_SRC WHERE OPB_SRC.SRC_ID = OPB_SEBL_LINKX.SRC_ID AND OPB_SRC.IS_VISIBLE <> 2 AND OPB_SRC.VERSION_NUMBER = OPB_SEBL_LINKX.VERSION_NUMBER AND OPB_SRC.SUBJ_ID = OPB_SEBL_LINKX.SUBJECT_ID AND OPB_SRC.SUBJ_ID = 178
2024-07-29 17:02:02.149 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73212_C
2024-07-29 17:02:02.149 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73212_C
2024-07-29 17:02:02.150 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73213_C CURSOR FOR SELECT COUNT(*) FROM OPB_VERSION_PROPS , OPB_SRC WHERE OPB_SRC.VERSION_NUMBER = OPB_VERSION_PROPS.VERSION_NUMBER AND OPB_SRC.SRC_ID = OPB_VERSION_PROPS.OBJECT_ID AND OPB_VERSION_PROPS.OBJECT_TYPE = 1 AND OPB_SRC.SUBJ_ID = OPB_VERSION_PROPS.SUBJECT_ID AND OPB_SRC.SUBJ_ID = 178 AND OPB_SRC.IS_VISIBLE <> 2 AND OPB_VERSION_PROPS.IS_SHORTCUT = 0
2024-07-29 17:02:02.151 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73213_C
2024-07-29 17:02:02.155 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73213_C
2024-07-29 17:02:02.156 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73214_C CURSOR FOR SELECT COUNT(*) FROM OPB_MD_CUBE WHERE SUBJECT_ID = 178 AND IS_VISIBLE <> 2
2024-07-29 17:02:02.157 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73214_C
2024-07-29 17:02:02.157 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73214_C
2024-07-29 17:02:02.159 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73215_C CURSOR FOR SELECT COUNT(*) FROM OPB_MD_DIMENSION WHERE SUBJECT_ID = 178 AND IS_VISIBLE <> 2
2024-07-29 17:02:02.159 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73215_C
2024-07-29 17:02:02.160 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73215_C
2024-07-29 17:02:02.161 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73216_C CURSOR FOR SELECT COUNT(*) FROM OPB_MD_LEVEL , OPB_MD_DIMENSION WHERE OPB_MD_LEVEL.VERSION_NUMBER = OPB_MD_DIMENSION.VERSION_NUMBER AND OPB_MD_LEVEL.DIMENSION_ID = OPB_MD_DIMENSION.DIMENSION_ID AND OPB_MD_DIMENSION.IS_VISIBLE <> 2 AND OPB_MD_LEVEL.SUBJECT_ID = OPB_MD_DIMENSION.SUBJECT_ID AND OPB_MD_LEVEL.SUBJECT_ID = 178
2024-07-29 17:02:02.162 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73216_C
2024-07-29 17:02:02.162 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73216_C
2024-07-29 17:02:02.163 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73217_C CURSOR FOR SELECT COUNT(*) FROM OPB_MD_LEVEL_FIELD , OPB_MD_DIMENSION WHERE OPB_MD_LEVEL_FIELD.DIMENSION_ID = OPB_MD_DIMENSION.DIMENSION_ID AND OPB_MD_LEVEL_FIELD.VERSION_NUMBER = OPB_MD_DIMENSION.VERSION_NUMBER AND OPB_MD_DIMENSION.IS_VISIBLE <> 2 AND OPB_MD_DIMENSION.SUBJECT_ID = 178
2024-07-29 17:02:02.167 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73217_C
2024-07-29 17:02:02.167 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73217_C
2024-07-29 17:02:02.169 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73218_C CURSOR FOR SELECT COUNT(*) FROM OPB_MD_HIERARCHY , OPB_MD_DIMENSION WHERE OPB_MD_HIERARCHY.VERSION_NUMBER=OPB_MD_DIMENSION.VERSION_NUMBER AND OPB_MD_HIERARCHY.DIMENSION_ID = OPB_MD_DIMENSION.DIMENSION_ID AND OPB_MD_DIMENSION.IS_VISIBLE <> 2 AND OPB_MD_DIMENSION.SUBJECT_ID = OPB_MD_HIERARCHY.SUBJECT_ID AND OPB_MD_DIMENSION.SUBJECT_ID = 178
2024-07-29 17:02:02.170 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73218_C
2024-07-29 17:02:02.170 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73218_C
2024-07-29 17:02:02.172 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73219_C CURSOR FOR SELECT COUNT(*) FROM OPB_MD_LVL_HRCHY , OPB_MD_DIMENSION WHERE OPB_MD_LVL_HRCHY.VERSION_NUMBER=OPB_MD_DIMENSION.VERSION_NUMBER AND OPB_MD_LVL_HRCHY.DIMENSION_ID = OPB_MD_DIMENSION.DIMENSION_ID AND OPB_MD_DIMENSION.IS_VISIBLE <> 2 AND OPB_MD_DIMENSION.SUBJECT_ID = 178
2024-07-29 17:02:02.173 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73219_C
2024-07-29 17:02:02.174 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73219_C
2024-07-29 17:02:02.176 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73220_C CURSOR FOR SELECT COUNT(*) FROM OPB_MD_HIER_GRAPH , OPB_MD_DIMENSION WHERE OPB_MD_HIER_GRAPH.DIMENSION_ID = OPB_MD_DIMENSION.DIMENSION_ID AND OPB_MD_HIER_GRAPH.VERSION_NUMBER = OPB_MD_DIMENSION.VERSION_NUMBER AND OPB_MD_DIMENSION.IS_VISIBLE <> 2 AND OPB_MD_DIMENSION.SUBJECT_ID = 178
2024-07-29 17:02:02.177 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73220_C
2024-07-29 17:02:02.177 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73220_C
2024-07-29 17:02:02.179 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73221_C CURSOR FOR SELECT COUNT(*) FROM OPB_VERSION_PROPS , OPB_MD_DIMENSION WHERE OPB_MD_DIMENSION.VERSION_NUMBER = OPB_VERSION_PROPS.VERSION_NUMBER AND OPB_MD_DIMENSION.DIMENSION_ID = OPB_VERSION_PROPS.OBJECT_ID AND OPB_VERSION_PROPS.OBJECT_TYPE = 31 AND OPB_MD_DIMENSION.SUBJECT_ID = OPB_VERSION_PROPS.SUBJECT_ID AND OPB_VERSION_PROPS.SUBJECT_ID = 178 AND OPB_MD_DIMENSION.IS_VISIBLE <> 2 AND OPB_VERSION_PROPS.IS_SHORTCUT = 0
2024-07-29 17:02:02.182 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73221_C
2024-07-29 17:02:02.183 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73221_C
2024-07-29 17:02:02.187 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73222_C CURSOR FOR SELECT COUNT(*) FROM OPB_MD_FACT , OPB_MD_CUBE WHERE OPB_MD_CUBE.CUBE_ID = OPB_MD_FACT.CUBE_ID AND OPB_MD_CUBE.VERSION_NUMBER = OPB_MD_FACT.VERSION_NUMBER AND OPB_MD_CUBE.IS_VISIBLE <> 2 AND OPB_MD_CUBE.SUBJECT_ID = OPB_MD_FACT.SUBJECT_ID AND OPB_MD_CUBE.SUBJECT_ID = 178
2024-07-29 17:02:02.189 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73222_C
2024-07-29 17:02:02.189 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73222_C
2024-07-29 17:02:02.192 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73223_C CURSOR FOR SELECT COUNT(*) FROM OPB_MD_CUBE_FIELD , OPB_MD_CUBE WHERE OPB_MD_CUBE_FIELD.CUBE_ID = OPB_MD_CUBE.CUBE_ID AND OPB_MD_CUBE.VERSION_NUMBER = OPB_MD_CUBE_FIELD.VERSION_NUMBER AND OPB_MD_CUBE.IS_VISIBLE <> 2 AND OPB_MD_CUBE.SUBJECT_ID = 178
2024-07-29 17:02:02.193 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73223_C
2024-07-29 17:02:02.194 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73223_C
2024-07-29 17:02:02.196 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73224_C CURSOR FOR SELECT COUNT(*) FROM OPB_MD_FACT_LEVEL , OPB_MD_CUBE WHERE OPB_MD_FACT_LEVEL.CUBE_ID = OPB_MD_CUBE.CUBE_ID AND OPB_MD_CUBE.VERSION_NUMBER = OPB_MD_FACT_LEVEL.VERSION_NUMBER AND OPB_MD_CUBE.IS_VISIBLE <> 2 AND OPB_MD_CUBE.SUBJECT_ID = 178
2024-07-29 17:02:02.196 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73224_C
2024-07-29 17:02:02.197 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73224_C
2024-07-29 17:02:02.198 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73225_C CURSOR FOR SELECT COUNT(*) FROM OPB_MD_CUBE_HRCHY , OPB_MD_CUBE WHERE OPB_MD_CUBE_HRCHY.CUBE_ID = OPB_MD_CUBE.CUBE_ID AND OPB_MD_CUBE.VERSION_NUMBER = OPB_MD_CUBE_HRCHY.VERSION_NUMBER AND OPB_MD_CUBE.IS_VISIBLE <> 2 AND OPB_MD_CUBE.SUBJECT_ID = 178
2024-07-29 17:02:02.199 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73225_C
2024-07-29 17:02:02.199 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73225_C
2024-07-29 17:02:02.201 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73226_C CURSOR FOR SELECT COUNT(*) FROM OPB_MD_FACT_FIELD , OPB_MD_CUBE WHERE OPB_MD_FACT_FIELD.CUBE_ID = OPB_MD_CUBE.CUBE_ID AND OPB_MD_CUBE.VERSION_NUMBER = OPB_MD_FACT_FIELD.VERSION_NUMBER AND OPB_MD_CUBE.IS_VISIBLE <> 2 AND OPB_MD_CUBE.SUBJECT_ID = 178
2024-07-29 17:02:02.202 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73226_C
2024-07-29 17:02:02.202 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73226_C
2024-07-29 17:02:02.203 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73227_C CURSOR FOR SELECT COUNT(*) FROM OPB_VERSION_PROPS , OPB_MD_CUBE WHERE OPB_MD_CUBE.VERSION_NUMBER = OPB_VERSION_PROPS.VERSION_NUMBER AND OPB_MD_CUBE.CUBE_ID = OPB_VERSION_PROPS.OBJECT_ID AND OPB_VERSION_PROPS.OBJECT_TYPE = 30 AND OPB_MD_CUBE.SUBJECT_ID = OPB_VERSION_PROPS.SUBJECT_ID AND OPB_VERSION_PROPS.SUBJECT_ID = 178 AND OPB_MD_CUBE.IS_VISIBLE <> 2 AND OPB_VERSION_PROPS.IS_SHORTCUT = 0
2024-07-29 17:02:02.204 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73227_C
2024-07-29 17:02:02.205 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73227_C
2024-07-29 17:02:02.207 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73228_C CURSOR FOR SELECT COUNT(*) FROM OPB_SAP_FUNC WHERE SUBJECT_ID = 178
2024-07-29 17:02:02.208 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73228_C
2024-07-29 17:02:02.209 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73228_C
2024-07-29 17:02:02.210 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73229_C CURSOR FOR SELECT COUNT(*) FROM OPB_SAP_FUNC_PARAM WHERE SUBJECT_ID = 178
2024-07-29 17:02:02.210 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73229_C
2024-07-29 17:02:02.211 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73229_C
2024-07-29 17:02:02.212 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73230_C CURSOR FOR SELECT COUNT(*) FROM OPB_SAP_STRUCT WHERE SUBJECT_ID = 178
2024-07-29 17:02:02.213 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73230_C
2024-07-29 17:02:02.213 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73230_C
2024-07-29 17:02:02.214 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73231_C CURSOR FOR SELECT COUNT(*) FROM OPB_SAP_STRUCT_FLD WHERE SUBJECT_ID = 178
2024-07-29 17:02:02.215 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73231_C
2024-07-29 17:02:02.215 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73231_C
2024-07-29 17:02:02.216 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73232_C CURSOR FOR SELECT COUNT(*) FROM OPB_WIDGET WHERE SUBJECT_ID = 178 AND IS_VISIBLE <> 2
2024-07-29 17:02:02.217 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73232_C
2024-07-29 17:02:02.223 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: CLOSE PM_13828_ac92890_73232_C
2024-07-29 17:02:02.224 IST [3509265] nfs_102@crsqa/[unknown] LOG: statement: DECLARE PM_13828_ac92890_73233_C CURSOR FOR SELECT COUNT(*) FROM OPB_WIDGET_FIELD , OPB_WIDGET WHERE OPB_WIDGET_FIELD.WIDGET_ID = OPB_WIDGET.WIDGET_ID AND OPB_WIDGET.IS_VISIBLE <> 2 AND OPB_WIDGET_FIELD.VERSION_NUMBER = OPB_WIDGET.VERSION_NUMBER AND OPB_WIDGET.SUBJECT_ID = 178
2024-07-29 17:02:02.227 IST [3509265] nfs_102@crsqa/[unknown] LOG: execute <unnamed>: FETCH FORWARD 10000 FROM PM_13828_ac92890_73233_C
2024-07-29 17:02:02.236 IST [3483783] LOG: server process (PID 3501977) was terminated by signal 9: Killed
2024-07-29 17:02:02.236 IST [3483783] DETAIL: Failed process was running: SELECT NAME_SPACE, NAME, ID, STATUS, PRIVILEGES1, PRIVILEGES2, PRIVILEGES3, PRIVILEGES4, PRIVILEGES5 from OPB_USER_GROUP WHERE TYPE = 1 AND STATUS < $1 AND NAME = $2 AND NAME_SPACE = $3
2024-07-29 17:02:02.236 IST [3483783] LOG: terminating any other active server processes
2024-07-29 17:02:02.248 IST [3501834] nfs_102@crsqa/[unknown] LOG: could not send data to client: Connection reset by peer
2024-07-29 17:02:03.433 IST [3483783] LOG: all server processes terminated; reinitializing
2024-07-29 17:02:04.270 IST [3512902] LOG: database system was interrupted; last known up at 2024-07-29 17:02:01 IST
2024-07-29 17:02:06.037 IST [3512902] LOG: database system was not properly shut down; automatic recovery in progress
2024-07-29 17:02:06.058 IST [3512902] LOG: redo starts at 20/470B1190
2024-07-29 17:02:06.066 IST [3512902] LOG: unexpected pageaddr 20/250D0000 in WAL segment 000000010000002000000047, LSN 20/470D0000, offset 851968
2024-07-29 17:02:06.067 IST [3512902] LOG: redo done at 20/470CFEB0 system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s
2024-07-29 17:02:06.081 IST [3512904] LOG: checkpoint starting: end-of-recovery immediate wait
2024-07-29 17:02:06.158 IST [3512904] LOG: checkpoint complete: wrote 110 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.071 s, sync=0.003 s, total=0.079 s; sync files=27, longest=0.002 s, average=0.001 s; distance=123 kB, estimate=123 kB; lsn=20/470D0048, redo lsn=20/470D0048
2024-07-29 17:02:06.164 IST [3483783] LOG: database system is ready to accept connections
2024-07-29 17:02:16.653 IST [3513712] nfs_102@crsqa/[unknown] LOG: statement: select CURRENT_TIMESTAMP;
2024-07-29 17:02:16.679 IST [3513712] nfs_102@crsqa/[unknown] LOG: execute PM_15000_2c29920_1: SELECT REPVERSION FROM OPB_REPOSIT WHERE RECID = 2
2024-07-29 17:02:16.680 IST [3513712] nfs_102@crsqa/[unknown] LOG: statement: COMMIT
Postgres16.2_ServerLogs.txt (text/plain, 2.8K) download
[root@invoppsqldb03 ~]# cat /var/log/messages | grep postgres
Jul 29 12:16:38 invoppsqldb03 su[3483573]: (to postgres) root on pts/1
Jul 29 17:02:01 invoppsqldb03 kernel: [3483783] 26 3483783 4365039 88416 868352 0 0 postgres
Jul 29 17:02:01 invoppsqldb03 kernel: [3483784] 26 3483784 70372 456 102400 0 0 postgres
Jul 29 17:02:01 invoppsqldb03 kernel: [3483785] 26 3483785 4365205 509161 4251648 0 0 postgres
Jul 29 17:02:01 invoppsqldb03 kernel: [3483786] 26 3483786 4365084 34281 425984 0 0 postgres
Jul 29 17:02:01 invoppsqldb03 kernel: [3483788] 26 3483788 4365039 4620 155648 0 0 postgres
Jul 29 17:02:01 invoppsqldb03 kernel: [3483789] 26 3483789 4365466 812 167936 0 0 postgres
Jul 29 17:02:01 invoppsqldb03 kernel: [3483790] 26 3483790 4365453 745 139264 0 0 postgres
Jul 29 17:02:01 invoppsqldb03 kernel: [3501834] 26 3501834 7454356 3553444 28983296 0 0 postgres
Jul 29 17:02:01 invoppsqldb03 kernel: [3501836] 26 3501836 7251239 3348709 27348992 0 0 postgres
Jul 29 17:02:01 invoppsqldb03 kernel: [3501975] 26 3501975 7362364 3459745 28241920 0 0 postgres
Jul 29 17:02:01 invoppsqldb03 kernel: [3501976] 26 3501976 7322236 3420160 27926528 0 0 postgres
Jul 29 17:02:01 invoppsqldb03 kernel: [3501977] 26 3501977 7466146 3558503 29077504 0 0 postgres
Jul 29 17:02:01 invoppsqldb03 kernel: [3509265] 26 3509265 4674654 765307 6701056 0 0 postgres
Jul 29 17:02:01 invoppsqldb03 kernel: [3510945] 26 3510945 4365738 4521 364544 0 0 postgres
Jul 29 17:02:01 invoppsqldb03 kernel: [3510963] 26 3510963 4366862 5781 372736 0 0 postgres
Jul 29 17:02:01 invoppsqldb03 kernel: [3511025] 26 3511025 4365220 1065 180224 0 0 postgres
Jul 29 17:02:01 invoppsqldb03 kernel: [3511027] 26 3511027 4365476 969 180224 0 0 postgres
Jul 29 17:02:01 invoppsqldb03 kernel: [3511031] 26 3511031 4365114 489 126976 0 0 postgres
Jul 29 17:02:01 invoppsqldb03 kernel: [3511033] 26 3511033 4365363 585 131072 0 0 postgres
Jul 29 17:02:01 invoppsqldb03 kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0-1,global_oom,task_memcg=/user.slice/user-0.slice/session-1097.scope,task=postgres,pid=3501977,uid=26
Jul 29 17:02:01 invoppsqldb03 kernel: Out of memory: Killed process 3501977 (postgres) total-vm:29864584kB, anon-rss:12379920kB, file-rss:12kB, shmem-rss:1854080kB, UID:26 pgtables:28396kB oom_score_adj:0
* RE: BUG #18569: Memory leak in Postgres Enterprise server
2024-08-06 08:34 BUG #18569: Memory leak in Postgres Enterprise server PG Bug reporting form <noreply@postgresql.org>
2024-08-06 11:15 ` Re: BUG #18569: Memory leak in Postgres Enterprise server Daniel Gustafsson <daniel@yesql.se>
2024-08-06 12:01 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Sahu, Abhisek Kumar <absahu@informatica.com>
2024-08-07 05:35 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Mathias, Renci <rmathias@informatica.com>
@ 2024-08-07 11:14 ` Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>
2024-08-12 05:37 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Sahu, Abhisek Kumar <absahu@informatica.com>
0 siblings, 1 reply; 13+ messages in thread
From: Yeddula, Madhusudhan reddy [CONTINGENT WORKER] @ 2024-08-07 11:14 UTC (permalink / raw)
To: Mathias, Renci <rmathias@informatica.com>; Daniel Gustafsson <daniel@yesql.se>; pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>; Sahu, Abhisek Kumar <absahu@informatica.com>; Sachan, Vishal <vsachan@informatica.com>; +Cc: Kumar, Gaurav <gaurkumar@informatica.com>; Nayak, Deepak Vaikunta <dnayak@informatica.com>
HI @Daniel Gustafsson
Your report mentions "Postgres Enterprise Server", are you running a product from a vendor like EDB
ANS : We are not using Postgres Enterprise Server from EDB.
Are you running postgres installed by yourself via a package manager (or similar)?
Ans : Installed postgres myself through YUM menthod
If you are buying postgres from a vendor then please contact them, if not you are in the right place.
We did not buy postgres from any vendor .
NOTE : We are using opensource postgresql 16.2 server on RHEL9.
Madhusudhan | GOC Database Engineer
Informatica Global Operation Center
Office: Mob: +91 9502034172
Email: DL_GOC_Database_Services@informatica.com
-----Original Message-----
From: Mathias, Renci <rmathias@informatica.com>
Sent: Wednesday, August 7, 2024 11:05 AM
To: Daniel Gustafsson <daniel@yesql.se>; pgsql-bugs@lists.postgresql.org; Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>; Sahu, Abhisek Kumar <absahu@informatica.com>; Sachan, Vishal <vsachan@informatica.com>
Cc: Kumar, Gaurav <gaurkumar@informatica.com>
Subject: RE: BUG #18569: Memory leak in Postgres Enterprise server
Hi @Daniel,
The log files are attached for your reference.
Please let us know if you have any queries.
Thanks,
Renci Monthi Mathias | QA Engineer
Bagmane Tech Park, C V Raman Nagar, Bangalore - 560093
Email: rmathias@informatica.com
-----Original Message-----
From: Sahu, Abhisek Kumar <absahu@informatica.com>
Sent: Tuesday, August 6, 2024 5:31 PM
To: Daniel Gustafsson <daniel@yesql.se>; pgsql-bugs@lists.postgresql.org; Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>; Sachan, Vishal <vsachan@informatica.com>
Cc: Mathias, Renci <rmathias@informatica.com>; Kumar, Gaurav <gaurkumar@informatica.com>
Subject: RE: BUG #18569: Memory leak in Postgres Enterprise server
Hi Daniel,
Thank you for your response.
@Sachan, Vishal please attach all the logs, including the ODL tracing.
Adding @Yeddula, Madhusudhan reddy [CONTINGENT WORKER] DBA to answer the below queries.
Regards
Abhisek
-----Original Message-----
From: Daniel Gustafsson <daniel@yesql.se>
Sent: Tuesday, August 6, 2024 4:45 PM
To: Sahu, Abhisek Kumar <absahu@informatica.com>; pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #18569: Memory leak in Postgres Enterprise server
Caution: This email originated from outside of the organization. Review for Phishing!
> On 6 Aug 2024, at 10:34, PG Bug reporting form <noreply@postgresql.org> wrote:
> PostgreSQL Server/Client Version: PostgreSQL 16.2
Your report mentions "Postgres Enterprise Server", are you running a product from a vendor like EDB or are you running postgres installed by yourself via a package manager (or similar)? If you are buying postgres from a vendor then please contact them, if not you are in the right place.
Upgrading to the latest 16.3 is also recommended even though there are no references to memory leak fixes in the release notes.
> We have also attached our Repository logs, PostgreSQL server logs and
> DB logs.
No, there are no logs attached (I don't think the bugreporting form support attaching any files). Please provide additional information as there is nothing to go in the report.
--
Daniel Gustafsson
^ permalink raw reply [nested|flat] 13+ messages in thread
* RE: BUG #18569: Memory leak in Postgres Enterprise server
2024-08-06 08:34 BUG #18569: Memory leak in Postgres Enterprise server PG Bug reporting form <noreply@postgresql.org>
2024-08-06 11:15 ` Re: BUG #18569: Memory leak in Postgres Enterprise server Daniel Gustafsson <daniel@yesql.se>
2024-08-06 12:01 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Sahu, Abhisek Kumar <absahu@informatica.com>
2024-08-07 05:35 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Mathias, Renci <rmathias@informatica.com>
2024-08-07 11:14 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>
@ 2024-08-12 05:37 ` Sahu, Abhisek Kumar <absahu@informatica.com>
2024-08-12 08:44 ` Re: BUG #18569: Memory leak in Postgres Enterprise server Tomas Vondra <tomas@vondra.me>
2024-08-14 08:52 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>
0 siblings, 2 replies; 13+ messages in thread
From: Sahu, Abhisek Kumar @ 2024-08-12 05:37 UTC (permalink / raw)
To: Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>; Mathias, Renci <rmathias@informatica.com>; Daniel Gustafsson <daniel@yesql.se>; pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>; Sachan, Vishal <vsachan@informatica.com>; +Cc: Kumar, Gaurav <gaurkumar@informatica.com>; Nayak, Deepak Vaikunta <dnayak@informatica.com>
Hi Daniel,
Kindly let us know if you have any updates.
Regards
Abhisek Kumar Sahu
Senior Manager
Main: +91 80 40203000 Ext : 3135
Mobile: 9886680490
Email: absahu@informatica.com
-----Original Message-----
From: Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>
Sent: Wednesday, August 7, 2024 4:45 PM
To: Mathias, Renci <rmathias@informatica.com>; Daniel Gustafsson <daniel@yesql.se>; pgsql-bugs@lists.postgresql.org; Sahu, Abhisek Kumar <absahu@informatica.com>; Sachan, Vishal <vsachan@informatica.com>
Cc: Kumar, Gaurav <gaurkumar@informatica.com>; Nayak, Deepak Vaikunta <dnayak@informatica.com>
Subject: RE: BUG #18569: Memory leak in Postgres Enterprise server
HI @Daniel Gustafsson
Your report mentions "Postgres Enterprise Server", are you running a product from a vendor like EDB
ANS : We are not using Postgres Enterprise Server from EDB.
Are you running postgres installed by yourself via a package manager (or similar)?
Ans : Installed postgres myself through YUM menthod
If you are buying postgres from a vendor then please contact them, if not you are in the right place.
We did not buy postgres from any vendor .
NOTE : We are using opensource postgresql 16.2 server on RHEL9.
Madhusudhan | GOC Database Engineer
Informatica Global Operation Center
Office: Mob: +91 9502034172
Email: DL_GOC_Database_Services@informatica.com
-----Original Message-----
From: Mathias, Renci <rmathias@informatica.com>
Sent: Wednesday, August 7, 2024 11:05 AM
To: Daniel Gustafsson <daniel@yesql.se>; pgsql-bugs@lists.postgresql.org; Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>; Sahu, Abhisek Kumar <absahu@informatica.com>; Sachan, Vishal <vsachan@informatica.com>
Cc: Kumar, Gaurav <gaurkumar@informatica.com>
Subject: RE: BUG #18569: Memory leak in Postgres Enterprise server
Hi @Daniel,
The log files are attached for your reference.
Please let us know if you have any queries.
Thanks,
Renci Monthi Mathias | QA Engineer
Bagmane Tech Park, C V Raman Nagar, Bangalore - 560093
Email: rmathias@informatica.com
-----Original Message-----
From: Sahu, Abhisek Kumar <absahu@informatica.com>
Sent: Tuesday, August 6, 2024 5:31 PM
To: Daniel Gustafsson <daniel@yesql.se>; pgsql-bugs@lists.postgresql.org; Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>; Sachan, Vishal <vsachan@informatica.com>
Cc: Mathias, Renci <rmathias@informatica.com>; Kumar, Gaurav <gaurkumar@informatica.com>
Subject: RE: BUG #18569: Memory leak in Postgres Enterprise server
Hi Daniel,
Thank you for your response.
@Sachan, Vishal please attach all the logs, including the ODL tracing.
Adding @Yeddula, Madhusudhan reddy [CONTINGENT WORKER] DBA to answer the below queries.
Regards
Abhisek
-----Original Message-----
From: Daniel Gustafsson <daniel@yesql.se>
Sent: Tuesday, August 6, 2024 4:45 PM
To: Sahu, Abhisek Kumar <absahu@informatica.com>; pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #18569: Memory leak in Postgres Enterprise server
Caution: This email originated from outside of the organization. Review for Phishing!
> On 6 Aug 2024, at 10:34, PG Bug reporting form <noreply@postgresql.org> wrote:
> PostgreSQL Server/Client Version: PostgreSQL 16.2
Your report mentions "Postgres Enterprise Server", are you running a product from a vendor like EDB or are you running postgres installed by yourself via a package manager (or similar)? If you are buying postgres from a vendor then please contact them, if not you are in the right place.
Upgrading to the latest 16.3 is also recommended even though there are no references to memory leak fixes in the release notes.
> We have also attached our Repository logs, PostgreSQL server logs and
> DB logs.
No, there are no logs attached (I don't think the bugreporting form support attaching any files). Please provide additional information as there is nothing to go in the report.
--
Daniel Gustafsson
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: BUG #18569: Memory leak in Postgres Enterprise server
2024-08-06 08:34 BUG #18569: Memory leak in Postgres Enterprise server PG Bug reporting form <noreply@postgresql.org>
2024-08-06 11:15 ` Re: BUG #18569: Memory leak in Postgres Enterprise server Daniel Gustafsson <daniel@yesql.se>
2024-08-06 12:01 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Sahu, Abhisek Kumar <absahu@informatica.com>
2024-08-07 05:35 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Mathias, Renci <rmathias@informatica.com>
2024-08-07 11:14 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>
2024-08-12 05:37 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Sahu, Abhisek Kumar <absahu@informatica.com>
@ 2024-08-12 08:44 ` Tomas Vondra <tomas@vondra.me>
1 sibling, 0 replies; 13+ messages in thread
From: Tomas Vondra @ 2024-08-12 08:44 UTC (permalink / raw)
To: Sahu, Abhisek Kumar <absahu@informatica.com>; Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>; Mathias, Renci <rmathias@informatica.com>; Daniel Gustafsson <daniel@yesql.se>; pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>; Sachan, Vishal <vsachan@informatica.com>; +Cc: Kumar, Gaurav <gaurkumar@informatica.com>; Nayak, Deepak Vaikunta <dnayak@informatica.com>
Hi Abhisek,
On 8/12/24 07:37, Sahu, Abhisek Kumar wrote:
> Hi Daniel,
>
> Kindly let us know if you have any updates.
>
It's hard to say what happened, based on the information provided so
far. It might be that there really is a bug / memory leak, or maybe the
selected query plan is not great, or maybe there's a configuration
issue. We'll need more information to find out.
1) What query was the failed backend running? Was it a big long-running
query (e.g. in ETL/BI application), or many small OLTP queries?
2) Do you know what execution plan was used?
3) If you rerun the query, does the memory usage grow the same way? This
would allow us to collect important data. For example, it would be very
useful if you could attach a debugger to a backend consuming a lot of
memory, and get memory context stats.
4) Can you share the configuration parameters? I think the memory config
is the most interesting one (shared_buffers, work_mem, ...). How much
RAM does the system have?
5) Is this a parallel query? In the snippet from /var/log/messages there
are 3-5 postgres processes that look like they might be parallel workers
(nearly sequential PIDs, same memory pattern).
6) Is this a long-running problem, or did it just start to happen
recently? Perhaps after some change/upgrade?
regards
--
Tomas Vondra
^ permalink raw reply [nested|flat] 13+ messages in thread
* RE: BUG #18569: Memory leak in Postgres Enterprise server
2024-08-06 08:34 BUG #18569: Memory leak in Postgres Enterprise server PG Bug reporting form <noreply@postgresql.org>
2024-08-06 11:15 ` Re: BUG #18569: Memory leak in Postgres Enterprise server Daniel Gustafsson <daniel@yesql.se>
2024-08-06 12:01 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Sahu, Abhisek Kumar <absahu@informatica.com>
2024-08-07 05:35 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Mathias, Renci <rmathias@informatica.com>
2024-08-07 11:14 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>
2024-08-12 05:37 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Sahu, Abhisek Kumar <absahu@informatica.com>
@ 2024-08-14 08:52 ` Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>
2024-08-19 07:19 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Mathias, Renci <rmathias@informatica.com>
1 sibling, 1 reply; 13+ messages in thread
From: Yeddula, Madhusudhan reddy [CONTINGENT WORKER] @ 2024-08-14 08:52 UTC (permalink / raw)
To: Sahu, Abhisek Kumar <absahu@informatica.com>; Mathias, Renci <rmathias@informatica.com>; Daniel Gustafsson <daniel@yesql.se>; pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>; Sachan, Vishal <vsachan@informatica.com>; +Cc: Kumar, Gaurav <gaurkumar@informatica.com>; Nayak, Deepak Vaikunta <dnayak@informatica.com>
HI @Sahu, Abhisek Kumar
Have done analysis from my end and changed postgres parameters in postgresql.conf file regarding Memory/CPU and other parameters as well. Kindly re run the repo and let me know if it is getting failed with same error.
Madhusudhan | GOC Database Engineer
Informatica Global Operation Center
Office: Mob: +91 9502034172
Email: DL_GOC_Database_Services@informatica.com
-----Original Message-----
From: Sahu, Abhisek Kumar <absahu@informatica.com>
Sent: Monday, August 12, 2024 11:08 AM
To: Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>; Mathias, Renci <rmathias@informatica.com>; Daniel Gustafsson <daniel@yesql.se>; pgsql-bugs@lists.postgresql.org; Sachan, Vishal <vsachan@informatica.com>
Cc: Kumar, Gaurav <gaurkumar@informatica.com>; Nayak, Deepak Vaikunta <dnayak@informatica.com>
Subject: RE: BUG #18569: Memory leak in Postgres Enterprise server
Hi Daniel,
Kindly let us know if you have any updates.
Regards
Abhisek Kumar Sahu
Senior Manager
Main: +91 80 40203000 Ext : 3135
Mobile: 9886680490
Email: absahu@informatica.com
-----Original Message-----
From: Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>
Sent: Wednesday, August 7, 2024 4:45 PM
To: Mathias, Renci <rmathias@informatica.com>; Daniel Gustafsson <daniel@yesql.se>; pgsql-bugs@lists.postgresql.org; Sahu, Abhisek Kumar <absahu@informatica.com>; Sachan, Vishal <vsachan@informatica.com>
Cc: Kumar, Gaurav <gaurkumar@informatica.com>; Nayak, Deepak Vaikunta <dnayak@informatica.com>
Subject: RE: BUG #18569: Memory leak in Postgres Enterprise server
HI @Daniel Gustafsson
Your report mentions "Postgres Enterprise Server", are you running a product from a vendor like EDB
ANS : We are not using Postgres Enterprise Server from EDB.
Are you running postgres installed by yourself via a package manager (or similar)?
Ans : Installed postgres myself through YUM menthod
If you are buying postgres from a vendor then please contact them, if not you are in the right place.
We did not buy postgres from any vendor .
NOTE : We are using opensource postgresql 16.2 server on RHEL9.
Madhusudhan | GOC Database Engineer
Informatica Global Operation Center
Office: Mob: +91 9502034172
Email: DL_GOC_Database_Services@informatica.com
-----Original Message-----
From: Mathias, Renci <rmathias@informatica.com>
Sent: Wednesday, August 7, 2024 11:05 AM
To: Daniel Gustafsson <daniel@yesql.se>; pgsql-bugs@lists.postgresql.org; Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>; Sahu, Abhisek Kumar <absahu@informatica.com>; Sachan, Vishal <vsachan@informatica.com>
Cc: Kumar, Gaurav <gaurkumar@informatica.com>
Subject: RE: BUG #18569: Memory leak in Postgres Enterprise server
Hi @Daniel,
The log files are attached for your reference.
Please let us know if you have any queries.
Thanks,
Renci Monthi Mathias | QA Engineer
Bagmane Tech Park, C V Raman Nagar, Bangalore - 560093
Email: rmathias@informatica.com
-----Original Message-----
From: Sahu, Abhisek Kumar <absahu@informatica.com>
Sent: Tuesday, August 6, 2024 5:31 PM
To: Daniel Gustafsson <daniel@yesql.se>; pgsql-bugs@lists.postgresql.org; Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>; Sachan, Vishal <vsachan@informatica.com>
Cc: Mathias, Renci <rmathias@informatica.com>; Kumar, Gaurav <gaurkumar@informatica.com>
Subject: RE: BUG #18569: Memory leak in Postgres Enterprise server
Hi Daniel,
Thank you for your response.
@Sachan, Vishal please attach all the logs, including the ODL tracing.
Adding @Yeddula, Madhusudhan reddy [CONTINGENT WORKER] DBA to answer the below queries.
Regards
Abhisek
-----Original Message-----
From: Daniel Gustafsson <daniel@yesql.se>
Sent: Tuesday, August 6, 2024 4:45 PM
To: Sahu, Abhisek Kumar <absahu@informatica.com>; pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #18569: Memory leak in Postgres Enterprise server
Caution: This email originated from outside of the organization. Review for Phishing!
> On 6 Aug 2024, at 10:34, PG Bug reporting form <noreply@postgresql.org> wrote:
> PostgreSQL Server/Client Version: PostgreSQL 16.2
Your report mentions "Postgres Enterprise Server", are you running a product from a vendor like EDB or are you running postgres installed by yourself via a package manager (or similar)? If you are buying postgres from a vendor then please contact them, if not you are in the right place.
Upgrading to the latest 16.3 is also recommended even though there are no references to memory leak fixes in the release notes.
> We have also attached our Repository logs, PostgreSQL server logs and
> DB logs.
No, there are no logs attached (I don't think the bugreporting form support attaching any files). Please provide additional information as there is nothing to go in the report.
--
Daniel Gustafsson
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: BUG #18569: Memory leak in Postgres Enterprise server
2024-08-06 08:34 BUG #18569: Memory leak in Postgres Enterprise server PG Bug reporting form <noreply@postgresql.org>
2024-08-06 11:15 ` Re: BUG #18569: Memory leak in Postgres Enterprise server Daniel Gustafsson <daniel@yesql.se>
2024-08-06 12:01 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Sahu, Abhisek Kumar <absahu@informatica.com>
2024-08-07 05:35 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Mathias, Renci <rmathias@informatica.com>
2024-08-07 11:14 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>
2024-08-12 05:37 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Sahu, Abhisek Kumar <absahu@informatica.com>
2024-08-14 08:52 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>
2024-08-19 07:19 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Mathias, Renci <rmathias@informatica.com>
@ 2024-08-19 14:06 ` Tomas Vondra <tomas@vondra.me>
1 sibling, 0 replies; 13+ messages in thread
From: Tomas Vondra @ 2024-08-19 14:06 UTC (permalink / raw)
To: Mathias, Renci <rmathias@informatica.com>; Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>; Sahu, Abhisek Kumar <absahu@informatica.com>; Daniel Gustafsson <daniel@yesql.se>; pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>; Sachan, Vishal <vsachan@informatica.com>; +Cc: Kumar, Gaurav <gaurkumar@informatica.com>; Nayak, Deepak Vaikunta <dnayak@informatica.com>
On 8/19/24 09:19, Mathias, Renci wrote:
> Hi @Yeddula, Madhusudhan reddy <mailto:myeddula@informatica.com>,
>
> Post re-running the operations, the repository experienced downtime
> after 32 hours and the PID was subsequently changed.
>
>
>
> Here are the details of the run:
>
> * *Run Start Time:* 14th Aug 8:00 PM
> * *Operation Failure:* 15th Aug 8:21 AM (The repository PID 16320 did
> not change and the operation hang)
> * *All Operations Hang:* 16th Aug 10:38 AM
> * *Repository PID Change:* 16th Aug 11:01 AM (PID changed to 10544)
>
>
>
> Below is the screenshot for your reference with reference to Operation
> Failure*:* 15th Aug 8:21 AM
>
>
> Attached is the file with reference to Repository Logs.
>
> Could you please investigate the DB server logs on your end and address
> the queries raised by the PostgreSQL team? Your prompt attention to this
> issue would be of great help.
>
I'm sorry, but those errors / logs are apparently from some proprietary
Informatica product. I doubt anyone here is going to trawl 7.5MB of
verbose logs in search of what might have gone wrong. It's not even
clear to me what should we be looking for and at what time. You'll have
to do that yourself and tell us what the interesting bits are.
In my previous reply from August 12 I asked for a number of things (what
query was running/failed, explain plan, info on memory usage, info on
config parameters, ...). AFAIK we got exactly zero of those responses.
I'm sorry, but that's pretty crucial information for issues like this. I
doubt anyone will be able to help you until you provide that.
regards
--
Tomas Vondra
^ permalink raw reply [nested|flat] 13+ messages in thread
* RE: BUG #18569: Memory leak in Postgres Enterprise server
2024-08-06 08:34 BUG #18569: Memory leak in Postgres Enterprise server PG Bug reporting form <noreply@postgresql.org>
2024-08-06 11:15 ` Re: BUG #18569: Memory leak in Postgres Enterprise server Daniel Gustafsson <daniel@yesql.se>
2024-08-06 12:01 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Sahu, Abhisek Kumar <absahu@informatica.com>
2024-08-07 05:35 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Mathias, Renci <rmathias@informatica.com>
2024-08-07 11:14 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>
2024-08-12 05:37 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Sahu, Abhisek Kumar <absahu@informatica.com>
2024-08-14 08:52 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>
2024-08-19 07:19 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Mathias, Renci <rmathias@informatica.com>
@ 2024-09-20 10:03 ` Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>
2024-09-21 00:23 ` Re: BUG #18569: Memory leak in Postgres Enterprise server Tomas Vondra <tomas@vondra.me>
1 sibling, 1 reply; 13+ messages in thread
From: Yeddula, Madhusudhan reddy [CONTINGENT WORKER] @ 2024-09-20 10:03 UTC (permalink / raw)
To: Mathias, Renci <rmathias@informatica.com>; Sahu, Abhisek Kumar <absahu@informatica.com>; Daniel Gustafsson <daniel@yesql.se>; pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>; Sachan, Vishal <vsachan@informatica.com>; +Cc: Kumar, Gaurav <gaurkumar@informatica.com>; Nayak, Deepak Vaikunta <dnayak@informatica.com>
HI @Daniel Gustafsson<mailto:daniel@yesql.se>
Postgres server is restarting multiple times when we run the use case with 3 JOBS. We have enough resources and never seen CPU and memory is gong high.
Please find Postgres parameters along with application logs.
MEMORY : 128GB
CPU : 32 Cores.
max_connections = 200
shared_buffers = 32GB
effective_cache_size = 96GB
maintenance_work_mem = 2GB
checkpoint_completion_target = 0.9
wal_buffers = 16MB
default_statistics_target = 100
random_page_cost = 1.1
effective_io_concurrency = 200
work_mem = 41943kB
huge_pages = try
min_wal_size = 1GB
max_wal_size = 4GB
max_worker_processes = 32
max_parallel_workers_per_gather = 4
max_parallel_workers = 32
max_parallel_maintenance_workers = 4
Madhusudhan | GOC Database Engineer
Informatica Global Operation Center
Office: Mob: +91 9502034172
Email: DL_GOC_Database_Services@informatica.com<mailto:DL_GOC_Database_Services@informatica.com>
[cid:image002.png@01DB0B71.B4D6AF20]<http://www.informatica.com/;
From: Mathias, Renci <rmathias@informatica.com>
Sent: Monday, August 19, 2024 12:50 PM
To: Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>; Sahu, Abhisek Kumar <absahu@informatica.com>; Daniel Gustafsson <daniel@yesql.se>; pgsql-bugs@lists.postgresql.org; Sachan, Vishal <vsachan@informatica.com>
Cc: Kumar, Gaurav <gaurkumar@informatica.com>; Nayak, Deepak Vaikunta <dnayak@informatica.com>
Subject: RE: BUG #18569: Memory leak in Postgres Enterprise server
Hi @Yeddula, Madhusudhan reddy<mailto:myeddula@informatica.com>,
Post re-running the operations, the repository experienced downtime after 32 hours and the PID was subsequently changed.
Here are the details of the run:
* Run Start Time: 14th Aug 8:00 PM
* Operation Failure: 15th Aug 8:21 AM (The repository PID 16320 did not change and the operation hang)
* All Operations Hang: 16th Aug 10:38 AM
* Repository PID Change: 16th Aug 11:01 AM (PID changed to 10544)
Below is the screenshot for your reference with reference to Operation Failure: 15th Aug 8:21 AM
[cid:image001.png@01DB0B6D.C4BF3E10]
Attached is the file with reference to Repository Logs.
Could you please investigate the DB server logs on your end and address the queries raised by the PostgreSQL team? Your prompt attention to this issue would be of great help.
Thanks,
Renci
-----Original Message-----
From: Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com<mailto:myeddula@informatica.com>>
Sent: Wednesday, August 14, 2024 2:22 PM
To: Sahu, Abhisek Kumar <absahu@informatica.com<mailto:absahu@informatica.com>>; Mathias, Renci <rmathias@informatica.com<mailto:rmathias@informatica.com>>; Daniel Gustafsson <daniel@yesql.se<mailto:daniel@yesql.se>>; pgsql-bugs@lists.postgresql.org<mailto:pgsql-bugs@lists.postgresql.org>; Sachan, Vishal <vsachan@informatica.com<mailto:vsachan@informatica.com>>
Cc: Kumar, Gaurav <gaurkumar@informatica.com<mailto:gaurkumar@informatica.com>>; Nayak, Deepak Vaikunta <dnayak@informatica.com<mailto:dnayak@informatica.com>>
Subject: RE: BUG #18569: Memory leak in Postgres Enterprise server
HI @Sahu, Abhisek Kumar
Have done analysis from my end and changed postgres parameters in postgresql.conf file regarding Memory/CPU and other parameters as well. Kindly re run the repo and let me know if it is getting failed with same error.
Madhusudhan | GOC Database Engineer
Informatica Global Operation Center
Office: Mob: +91 9502034172
Email: DL_GOC_Database_Services@informatica.com<mailto:DL_GOC_Database_Services@informatica.com>
-----Original Message-----
From: Sahu, Abhisek Kumar <absahu@informatica.com<mailto:absahu@informatica.com>>
Sent: Monday, August 12, 2024 11:08 AM
To: Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com<mailto:myeddula@informatica.com>>; Mathias, Renci <rmathias@informatica.com<mailto:rmathias@informatica.com>>; Daniel Gustafsson <daniel@yesql.se<mailto:daniel@yesql.se>>; pgsql-bugs@lists.postgresql.org<mailto:pgsql-bugs@lists.postgresql.org>; Sachan, Vishal <vsachan@informatica.com<mailto:vsachan@informatica.com>>
Cc: Kumar, Gaurav <gaurkumar@informatica.com<mailto:gaurkumar@informatica.com>>; Nayak, Deepak Vaikunta <dnayak@informatica.com<mailto:dnayak@informatica.com>>
Subject: RE: BUG #18569: Memory leak in Postgres Enterprise server
Hi Daniel,
Kindly let us know if you have any updates.
Regards
Abhisek Kumar Sahu
Senior Manager
Main: +91 80 40203000 Ext : 3135
Mobile: 9886680490
Email: absahu@informatica.com<mailto:absahu@informatica.com>
-----Original Message-----
From: Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com<mailto:myeddula@informatica.com>>
Sent: Wednesday, August 7, 2024 4:45 PM
To: Mathias, Renci <rmathias@informatica.com<mailto:rmathias@informatica.com>>; Daniel Gustafsson <daniel@yesql.se<mailto:daniel@yesql.se>>; pgsql-bugs@lists.postgresql.org<mailto:pgsql-bugs@lists.postgresql.org>; Sahu, Abhisek Kumar <absahu@informatica.com<mailto:absahu@informatica.com>>; Sachan, Vishal <vsachan@informatica.com<mailto:vsachan@informatica.com>>
Cc: Kumar, Gaurav <gaurkumar@informatica.com<mailto:gaurkumar@informatica.com>>; Nayak, Deepak Vaikunta <dnayak@informatica.com<mailto:dnayak@informatica.com>>
Subject: RE: BUG #18569: Memory leak in Postgres Enterprise server
HI @Daniel Gustafsson
Your report mentions "Postgres Enterprise Server", are you running a product from a vendor like EDB
ANS : We are not using Postgres Enterprise Server from EDB.
Are you running postgres installed by yourself via a package manager (or similar)?
Ans : Installed postgres myself through YUM menthod
If you are buying postgres from a vendor then please contact them, if not you are in the right place.
We did not buy postgres from any vendor .
NOTE : We are using opensource postgresql 16.2 server on RHEL9.
Madhusudhan | GOC Database Engineer
Informatica Global Operation Center
Office: Mob: +91 9502034172
Email: DL_GOC_Database_Services@informatica.com<mailto:DL_GOC_Database_Services@informatica.com>
-----Original Message-----
From: Mathias, Renci <rmathias@informatica.com<mailto:rmathias@informatica.com>>
Sent: Wednesday, August 7, 2024 11:05 AM
To: Daniel Gustafsson <daniel@yesql.se<mailto:daniel@yesql.se>>; pgsql-bugs@lists.postgresql.org<mailto:pgsql-bugs@lists.postgresql.org>; Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com<mailto:myeddula@informatica.com>>; Sahu, Abhisek Kumar <absahu@informatica.com<mailto:absahu@informatica.com>>; Sachan, Vishal <vsachan@informatica.com<mailto:vsachan@informatica.com>>
Cc: Kumar, Gaurav <gaurkumar@informatica.com<mailto:gaurkumar@informatica.com>>
Subject: RE: BUG #18569: Memory leak in Postgres Enterprise server
Hi @Daniel,
The log files are attached for your reference.
Please let us know if you have any queries.
Thanks,
Renci Monthi Mathias | QA Engineer
Bagmane Tech Park, C V Raman Nagar, Bangalore - 560093
Email: rmathias@informatica.com<mailto:rmathias@informatica.com>
-----Original Message-----
From: Sahu, Abhisek Kumar <absahu@informatica.com<mailto:absahu@informatica.com>>
Sent: Tuesday, August 6, 2024 5:31 PM
To: Daniel Gustafsson <daniel@yesql.se<mailto:daniel@yesql.se>>; pgsql-bugs@lists.postgresql.org<mailto:pgsql-bugs@lists.postgresql.org>; Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com<mailto:myeddula@informatica.com>>; Sachan, Vishal <vsachan@informatica.com<mailto:vsachan@informatica.com>>
Cc: Mathias, Renci <rmathias@informatica.com<mailto:rmathias@informatica.com>>; Kumar, Gaurav <gaurkumar@informatica.com<mailto:gaurkumar@informatica.com>>
Subject: RE: BUG #18569: Memory leak in Postgres Enterprise server
Hi Daniel,
Thank you for your response.
@Sachan, Vishal please attach all the logs, including the ODL tracing.
Adding @Yeddula, Madhusudhan reddy [CONTINGENT WORKER] DBA to answer the below queries.
Regards
Abhisek
-----Original Message-----
From: Daniel Gustafsson <daniel@yesql.se<mailto:daniel@yesql.se>>
Sent: Tuesday, August 6, 2024 4:45 PM
To: Sahu, Abhisek Kumar <absahu@informatica.com<mailto:absahu@informatica.com>>; pgsql-bugs@lists.postgresql.org<mailto:pgsql-bugs@lists.postgresql.org>
Subject: Re: BUG #18569: Memory leak in Postgres Enterprise server
Caution: This email originated from outside of the organization. Review for Phishing!
> On 6 Aug 2024, at 10:34, PG Bug reporting form <noreply@postgresql.org<mailto:noreply@postgresql.org>> wrote:
> PostgreSQL Server/Client Version: PostgreSQL 16.2
Your report mentions "Postgres Enterprise Server", are you running a product from a vendor like EDB or are you running postgres installed by yourself via a package manager (or similar)? If you are buying postgres from a vendor then please contact them, if not you are in the right place.
Upgrading to the latest 16.3 is also recommended even though there are no references to memory leak fixes in the release notes.
> We have also attached our Repository logs, PostgreSQL server logs and
> DB logs.
No, there are no logs attached (I don't think the bugreporting form support attaching any files). Please provide additional information as there is nothing to go in the report.
--
Daniel Gustafsson
Attachments:
image001.png (image/png, 50.3K) download
view image
image002.png (image/png, 7.3K) download
view image
* Re: BUG #18569: Memory leak in Postgres Enterprise server
2024-08-06 08:34 BUG #18569: Memory leak in Postgres Enterprise server PG Bug reporting form <noreply@postgresql.org>
2024-08-06 11:15 ` Re: BUG #18569: Memory leak in Postgres Enterprise server Daniel Gustafsson <daniel@yesql.se>
2024-08-06 12:01 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Sahu, Abhisek Kumar <absahu@informatica.com>
2024-08-07 05:35 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Mathias, Renci <rmathias@informatica.com>
2024-08-07 11:14 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>
2024-08-12 05:37 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Sahu, Abhisek Kumar <absahu@informatica.com>
2024-08-14 08:52 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>
2024-08-19 07:19 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Mathias, Renci <rmathias@informatica.com>
2024-09-20 10:03 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>
@ 2024-09-21 00:23 ` Tomas Vondra <tomas@vondra.me>
2024-09-21 00:28 ` Re: BUG #18569: Memory leak in Postgres Enterprise server Tomas Vondra <tomas@vondra.me>
0 siblings, 1 reply; 13+ messages in thread
From: Tomas Vondra @ 2024-09-21 00:23 UTC (permalink / raw)
To: Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>; Mathias, Renci <rmathias@informatica.com>; Sahu, Abhisek Kumar <absahu@informatica.com>; Daniel Gustafsson <daniel@yesql.se>; pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>; Sachan, Vishal <vsachan@informatica.com>; +Cc: Kumar, Gaurav <gaurkumar@informatica.com>; Nayak, Deepak Vaikunta <dnayak@informatica.com>
On 9/20/24 12:03, Yeddula, Madhusudhan reddy [CONTINGENT WORKER] wrote:
> HI @Daniel Gustafsson <mailto:daniel@yesql.se>
>
>
>
> Postgres server is restarting multiple times when we run the use case
> with 3 JOBS. We have enough resources and never seen CPU and memory is
> gong high.
>
This thread started by claim of a memory leak, but now you claim you
don't see CPU or memory usage going up? Or do I misunderstand?
If it is a memory leak, and the memory usage grows over time, you should
be able to monitor the system, see which backend(s) allocate the memory
and inspect what queries are being executed.
>
>
> Please find Postgres parameters along with application logs.
>
Application log is not very useful for investigating this, because it
only says this:
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
So, the database connection went away, but we're none the wiser why. We
don't even know if it was the query reported in the application log
causing the issue, it might have been some other process causing the
crash, and this query is just a victim.
You need to inspect the database log, which might tell you more - either
which backend actually crashed / why, possibly some additional info.
Also look at dmesg, which should have info about OOM killer, if that's
what killed one of the processes.
>
>
> MEMORY : 128GB
>
> CPU : 32 Cores.
>
>
>
>
>
> max_connections = 200
>
> shared_buffers = 32GB
>
> effective_cache_size = 96GB
>
> maintenance_work_mem = 2GB
>
> checkpoint_completion_target = 0.9
>
> wal_buffers = 16MB
>
> default_statistics_target = 100
>
> random_page_cost = 1.1
>
> effective_io_concurrency = 200
>
> work_mem = 41943kB
>
> huge_pages = try
>
> min_wal_size = 1GB
>
> max_wal_size = 4GB
>
> max_worker_processes = 32
>
> max_parallel_workers_per_gather = 4
>
> max_parallel_workers = 32
>
> max_parallel_maintenance_workers = 4
>
These parameters seem reasonable. I don't see any obvious problem.
regards
--
Tomas Vondra
^ permalink raw reply [nested|flat] 13+ messages in thread
* Re: BUG #18569: Memory leak in Postgres Enterprise server
2024-08-06 08:34 BUG #18569: Memory leak in Postgres Enterprise server PG Bug reporting form <noreply@postgresql.org>
2024-08-06 11:15 ` Re: BUG #18569: Memory leak in Postgres Enterprise server Daniel Gustafsson <daniel@yesql.se>
2024-08-06 12:01 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Sahu, Abhisek Kumar <absahu@informatica.com>
2024-08-07 05:35 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Mathias, Renci <rmathias@informatica.com>
2024-08-07 11:14 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>
2024-08-12 05:37 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Sahu, Abhisek Kumar <absahu@informatica.com>
2024-08-14 08:52 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>
2024-08-19 07:19 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Mathias, Renci <rmathias@informatica.com>
2024-09-20 10:03 ` RE: BUG #18569: Memory leak in Postgres Enterprise server Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>
2024-09-21 00:23 ` Re: BUG #18569: Memory leak in Postgres Enterprise server Tomas Vondra <tomas@vondra.me>
@ 2024-09-21 00:28 ` Tomas Vondra <tomas@vondra.me>
0 siblings, 0 replies; 13+ messages in thread
From: Tomas Vondra @ 2024-09-21 00:28 UTC (permalink / raw)
To: Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>; Mathias, Renci <rmathias@informatica.com>; Sahu, Abhisek Kumar <absahu@informatica.com>; Daniel Gustafsson <daniel@yesql.se>; pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>; Sachan, Vishal <vsachan@informatica.com>; +Cc: Kumar, Gaurav <gaurkumar@informatica.com>; Nayak, Deepak Vaikunta <dnayak@informatica.com>
On 9/21/24 02:23, Tomas Vondra wrote:
>
> ...
>
> Application log is not very useful for investigating this, because it
> only says this:
>
> server closed the connection unexpectedly
>
> This probably means the server terminated abnormally
>
> before or while processing the request.
>
> So, the database connection went away, but we're none the wiser why. We
> don't even know if it was the query reported in the application log
> causing the issue, it might have been some other process causing the
> crash, and this query is just a victim.
>
> You need to inspect the database log, which might tell you more - either
> which backend actually crashed / why, possibly some additional info.
> Also look at dmesg, which should have info about OOM killer, if that's
> what killed one of the processes.
>
BTW I noticed there's a separate "memory leak" report from you. Is that
related to the same database/instance?
--
Tomas Vondra
^ permalink raw reply [nested|flat] 13+ messages in thread
1 further message in this thread omitted from this page (render size limit).
Use the per-message pages, or the t.mbox.gz byte stream, for the complete thread.
end of thread, other threads:[~2024-09-21 00:28 UTC | newest]
Thread overview: 13+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2024-08-06 08:34 BUG #18569: Memory leak in Postgres Enterprise server PG Bug reporting form <noreply@postgresql.org>
2024-08-06 11:15 ` Daniel Gustafsson <daniel@yesql.se>
2024-08-06 12:01 ` Sahu, Abhisek Kumar <absahu@informatica.com>
2024-08-07 05:35 ` Mathias, Renci <rmathias@informatica.com>
2024-08-07 11:14 ` Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>
2024-08-12 05:37 ` Sahu, Abhisek Kumar <absahu@informatica.com>
2024-08-12 08:44 ` Tomas Vondra <tomas@vondra.me>
2024-08-14 08:52 ` Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>
2024-08-19 07:19 ` Mathias, Renci <rmathias@informatica.com>
2024-08-19 14:06 ` Tomas Vondra <tomas@vondra.me>
2024-09-20 10:03 ` Yeddula, Madhusudhan reddy [CONTINGENT WORKER] <myeddula@informatica.com>
2024-09-21 00:23 ` Tomas Vondra <tomas@vondra.me>
2024-09-21 00:28 ` Tomas Vondra <tomas@vondra.me>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox