Received: from giasbg01.vsnl.net.in (giasbg01.vsnl.net.in [202.54.12.17]) by hub.org (8.9.3/8.9.3) with ESMTP id CAA79243; Sat, 11 Dec 1999 02:06:07 -0500 (EST) (envelope-from diffs@vsnl.com) Received: from nagma (PPP-190-209.bng.vsnl.net.in [203.197.190.209]) by giasbg01.vsnl.net.in (8.9.3/8.9.3) with SMTP id MAA23270; Sat, 11 Dec 1999 12:36:53 +0500 (GMT+0500) Message-ID: <001901bf43a6$16f12dc0$dcbec5cb@nagma> From: "Differentiated Software Solutions Pvt. Ltd." To: , , Cc: "Kimi" Subject: Spin lock error Date: Sat, 11 Dec 1999 12:33:31 +0530 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_0016_01BF43D3.EF61FCC0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 4.72.2106.4 X-MimeOLE: Produced By Microsoft MimeOLE V4.72.2106.4 This is a multi-part message in MIME format. ------=_NextPart_000_0016_01BF43D3.EF61FCC0 Content-Type: text/plain; charset="utf-7" Content-Transfer-Encoding: 7bit Hi, We are getting spinlock errors under the following conditions Background : We have developed a high performance application using postgres database ( a 3rd party application server). This application should be scalanble to server 50 requests a second. Application is written in perl 5.005 using DBI-1.12, DBD-Pg 0.92 and Redhat linux 6.0 A request consists of one singleton select on a table of 200,000 rows on it's primary key, a heavy duty cursor which is a 4 table join but which returns 100 rows from tables containing maximum of 250 rows, a light sql opening a cursor on a table containing 500 rows, 2 inserts into tables which don't have primary key. We are connecting to the database with autoconnect off. Problem description : On a light load there are no problems. As we have increased the load on the application, the requests get piled up. We keep monitoring the requests getting piled up using ps ax +AHw- grep post. Somewhere when the requests crosses 20(ps returns more than 20 rows).... suddenly postmaster comes down +ACEAIQ- In the server log the following message is written out. FATAL: s+AF8-lock(4000b011) at spin.c:125, stuck spinlock. Aborting. FATAL: s+AF8-lock(4000b011) at spin.c:125, stuck spinlock. Aborting. FATAL: s+AF8-lock(4000b011) at spin.c:125, stuck spinlock. Aborting. FATAL: s+AF8-lock(4000b011) at spin.c:125, stuck spinlock. Aborting. FATAL: s+AF8-lock(40179f44) at bufmgr.c:490, stuck spinlock. Aborting. After this postmaster shutdown. One observation If we try to recreate this in another linux machine... multiple postmasters run in the linux machine when the number of requests become more. I'm also attaching a file containing 'ps' at the instance postmaster crashed. Can anybody please help Thanks and regards, Murali Differentiated Software Solutions Pvt. Ltd., 176, Gr. Floor, 6th Main 2nd Block RT Nagar Bangalore - 560 032 India Ph: 91 80 3431470 ------=_NextPart_000_0016_01BF43D3.EF61FCC0 Content-Type: text/plain; name="ps.txt" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="ps.txt" 6155 p0 S 0:00 grep post 4957 p2 S 0:00 /usr/local/bin/postmaster -i -p 5433 5992 p2 R 0:01 /usr/local/bin/postgres localhost postgres websight = COPY 6009 p2 S 0:01 /usr/local/bin/postgres localhost postgres adcept = commit 6020 p2 S 0:01 /usr/local/bin/postgres localhost postgres adcept = SELECT 6024 p2 S 0:01 /usr/local/bin/postgres localhost postgres adcept = SELECT 6038 p2 S 0:00 /usr/local/bin/postgres localhost postgres adcept = SELECT 6051 p2 S 0:00 /usr/local/bin/postgres localhost postgres adcept = commit 6062 p2 S 0:00 /usr/local/bin/postgres localhost postgres adcept = SELECT 6072 p2 S 0:00 /usr/local/bin/postgres localhost postgres adcept = SELECT 6076 p2 S 0:00 /usr/local/bin/postgres localhost postgres adcept = SELECT 6077 p2 R 0:00 /usr/local/bin/postgres localhost postgres adcept = idle 6078 p2 R 0:00 /usr/local/bin/postgres localhost postgres adcept = SELECT 6110 p2 S 0:00 /usr/local/bin/postgres localhost postgres adcept = SELECT 6112 p2 R 0:00 /usr/local/bin/postgres localhost postgres adcept = SELECT 6122 p2 S 0:00 /usr/local/bin/postgres localhost postgres adcept = SELECT 6137 p2 R 0:00 /usr/local/bin/postgres localhost postgres adcept = SELECT 6141 p2 R 0:00 /usr/local/bin/postgres localhost postgres adcept = SELECT 6147 p2 R 0:00 /usr/local/bin/postgres localhost postgres adcept = SELECT 6149 p2 R 0:00 /usr/local/bin/postgres localhost postgres adcept = idle 6151 p2 R 0:00 /usr/local/bin/postmaster -i 6165 p2 R 0:00 /usr/local/bin/postgres localhost postgres adcept = SELECT 6172 p2 R 0:00 /usr/local/bin/postgres localhost postgres adcept = SELECT 6184 p2 R 0:00 /usr/local/bin/postgres localhost postgres adcept = SELECT 20964 p6 S 0:00 su - postgres 23112 p4 S 0:00 su - postgres 27664 p2 R 0:03 /usr/local/bin/postmaster -i bash: /bin/ps: Too many open files in system=20 4957 p2 S 0:00 /usr/local/bin/postmaster -i -p 5433 8952 p2 S 0:01 /usr/local/bin/postgres localhost postgres adcept = idle 8958 p2 S 0:01 /usr/local/bin/postgres localhost postgres adcept = idle 8981 p2 S 0:00 /usr/local/bin/postgres localhost postgres websight = idle 8984 p2 S 0:01 /usr/local/bin/postgres localhost postgres adcept = SELECT 9012 p2 S 0:00 /usr/local/bin/postgres localhost postgres adcept = SELECT 9013 p2 S 0:00 /usr/local/bin/postgres localhost postgres adcept = idle 9052 p2 S 0:00 /usr/local/bin/postgres localhost postgres adcept = SELECT 9054 p2 S 0:00 /usr/local/bin/postgres localhost postgres adcept = SELECT 9066 p2 S 0:00 /usr/local/bin/postgres localhost postgres adcept = SELECT 9071 p2 S 0:00 /usr/local/bin/postgres localhost postgres adcept = idle 9077 p2 S 0:00 /usr/local/bin/postgres localhost postgres adcept = SELECT 9082 p2 S 0:00 /usr/local/bin/postgres localhost postgres adcept = SELECT 9083 p2 S 0:00 /usr/local/bin/postgres localhost postgres adcept = SELECT 9097 p2 S 0:00 /usr/local/bin/postgres localhost postgres adcept = SELECT 9119 p2 S 0:00 /usr/local/bin/postgres localhost postgres adcept = SELECT 9127 p2 S 0:00 /usr/local/bin/postgres localhost postgres adcept = SELECT 9128 p2 S 0:00 /usr/local/bin/postgres localhost postgres adcept = SELECT 9149 p2 S 0:00 /usr/local/bin/postgres localhost postgres adcept = SELECT 9155 p2 S 0:00 /usr/local/bin/postgres localhost postgres adcept = idle 9156 p2 S 0:00 /usr/local/bin/postgres localhost postgres adcept = idle 9158 p2 S 0:00 /usr/local/bin/postgres localhost postgres adcept = idle 9159 p2 S 0:00 /usr/local/bin/postgres localhost postgres adcept = idle 20964 p6 S 0:00 su - postgres 23112 p4 S 0:00 su - postgres ------=_NextPart_000_0016_01BF43D3.EF61FCC0--