Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1veKEs-00BuRa-19 for pgsql-performance@arkaria.postgresql.org; Fri, 09 Jan 2026 21:41:47 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.96) (envelope-from ) id 1veKEm-008h40-0z for pgsql-performance@arkaria.postgresql.org; Fri, 09 Jan 2026 21:41:41 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1veKEl-008h3r-1m for pgsql-performance@lists.postgresql.org; Fri, 09 Jan 2026 21:41:40 +0000 Received: from mx0b-002a6b01.pphosted.com ([67.231.153.219]) by makus.postgresql.org with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1veKEj-0058Pg-25 for pgsql-performance@lists.postgresql.org; Fri, 09 Jan 2026 21:41:38 +0000 Received: from pps.filterd (m0098284.ppops.net [127.0.0.1]) by mx0b-002a6b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 609H4WBJ028181 for ; Fri, 9 Jan 2026 21:41:36 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bestbuy.com; h= content-type:date:from:message-id:mime-version:subject:to; s= S1-20230630; bh=4PVNwneFYCWLiXrlZC9OYPhuzmCZ7KmMytka+Krk4Rk=; b= DdUn7XcJj4ZE0z3ZYA58lRrbA7zCp7+6dIiRAeeg99sn/r4+SSdYJpO83WgGtSZY 1SlfAYPeRkg8VZlsbtsjOe/zKDRaS9WU3d7oGszg93MOwe7wkCfWf6m2BSZTQ93u g7r0sk+fNaoBOGhVDfi1iOwju1HFA1cQP1LcuakXkUBJ8Z4r++McEEN2zyyP5qmo PAfb2Ln7ITUvChh9FxIF9twB9TG0H5aNqqvGrFKpOphCUfHEjRVVC0t7Sunqbt7u kGn5xjALJYqSH2mya9X+NjFmdeI0pjomKCxJzMa5ivzcgQ9Rimm7aAXVG+42eUiz BaV+eSUPeeTKfd7LlVoDPQ== Received: from ph7pr06cu001.outbound.protection.outlook.com (mail-westus3azon11010052.outbound.protection.outlook.com [52.101.201.52]) by mx0b-002a6b01.pphosted.com (PPS) with ESMTPS id 4bk5rmjwkd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 09 Jan 2026 21:41:36 +0000 (GMT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=qEFSM2N/pjRVmPkqpiOMJJ/fka81HEBHLgYYJ5p4e+MRhv/yCuy7aBiunhq3WjLTYQJfzcSNyTaQc5yUuDHIvaErbVPibB4A0aMjPbjmK/uRoCHyj8/zCKBRPLN/24E0fYNMPdeG8GvhEtu8sz0dQALo2Ne+5iswrR2dN5MCpVNNHUUBSavNaxI9ZfHCK3U8UQdHfIc7WMFiOgpW0gE1lDRnyBMMkVapdeUotPnFInjGRPIecoPdvRWni86tduu4htvXCmV+OI7XL2yvb9Nr45TUp9hDyYFVx8Xxq37G+UofySZI8lD3212S7dx0S/BJXGji7ukKnASm9xZQqZ+2eQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4PVNwneFYCWLiXrlZC9OYPhuzmCZ7KmMytka+Krk4Rk=; b=MctbBglCChmm/VO3cB98R8cv9IeOaIjufSbejAVIuVAwYSHQ+381f5bobwIoVlN0QvKkiamR1n5g98sw/AlNur6VmUMVwcNmQtWN0kk435up4r/FfGDfNSlu3IiYsoru8ioakkOYz0PkAk6TzhxwZAwArXYtT3gg8Z5tnkzfpkQROsHS2fD3FTEemPpOXRQB47Y1U2CCcKc6yLvasst3Dre9S2lrodERAzV0gWBCSaOGuNvGDm77NCmItfwWX04/GyT/ZG6A4/CwUBcnjOBvqgfIip5OOpj2omJzd1pHoJ3HNXWnUwqUUtsXowRGVoKwP6K2ViQZoZ+xpQewH/ZDrg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=bestbuy.com; dmarc=pass action=none header.from=bestbuy.com; dkim=pass header.d=bestbuy.com; arc=none Received: from BL0PR06MB5009.namprd06.prod.outlook.com (2603:10b6:208:66::19) by CH7PR06MB10596.namprd06.prod.outlook.com (2603:10b6:610:24b::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9499.2; Fri, 9 Jan 2026 21:41:33 +0000 Received: from BL0PR06MB5009.namprd06.prod.outlook.com ([fe80::e640:8c0e:6e38:77d2]) by BL0PR06MB5009.namprd06.prod.outlook.com ([fe80::e640:8c0e:6e38:77d2%7]) with mapi id 15.20.9456.013; Fri, 9 Jan 2026 21:41:33 +0000 From: "Dirschel, Steve-CW" To: "pgsql-performance@lists.postgresql.org" Subject: Troubleshooting a brief spike in active sessions Thread-Topic: Troubleshooting a brief spike in active sessions Thread-Index: AQHcga2DLxrFOfYCFUWjyu98zzUpoA== Date: Fri, 9 Jan 2026 21:41:33 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_e9c55f1f-f169-4db7-a264-a5084ccbb748_Enabled=True;MSIP_Label_e9c55f1f-f169-4db7-a264-a5084ccbb748_SiteId=135e8995-7d3b-4466-844b-a0d62ba5f495;MSIP_Label_e9c55f1f-f169-4db7-a264-a5084ccbb748_SetDate=2026-01-09T21:18:34.8285131Z;MSIP_Label_e9c55f1f-f169-4db7-a264-a5084ccbb748_Name=Confidential;MSIP_Label_e9c55f1f-f169-4db7-a264-a5084ccbb748_ContentBits=0;MSIP_Label_e9c55f1f-f169-4db7-a264-a5084ccbb748_Method=Standard x-ms-reactions: allow x-ms-publictraffictype: Email x-ms-traffictypediagnostic: BL0PR06MB5009:EE_|CH7PR06MB10596:EE_ x-ms-office365-filtering-correlation-id: 8d48defc-f507-476e-7703-08de4fc7dba5 x-proofpoint-outbound: 2ABD5B1121654D948E7788CED1FE3D92 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0;ARA:13230040|1800799024|376014|366016|38070700021|8096899003; x-microsoft-antispam-message-info: =?Windows-1252?Q?dTKbFxbtCtH2L06ZYeUBHcE7MqtQZd3s64VOmjfCbOig/hwDJ6QJAozu?= =?Windows-1252?Q?13VcU5pHD8lo/ItvjJ9Tnjp215ehLWXmoP8ClVhpgXidW3WvtBZcMyR8?= =?Windows-1252?Q?eF1lP49AtHl2I/aSc4kKDB9dU1Bva+PK+N99vxc5aRlwSA1gowqGuCto?= =?Windows-1252?Q?UO3A5wHJB7c187V3txYOuljpG7eEsSREU9KGu3a4WY7SWxWDUj1r6NMR?= =?Windows-1252?Q?qMDsdrYH8xTMQT+drmckaFoXp3iMkNepDpYpXjGrGLX9oNM7gkNCz2ZK?= =?Windows-1252?Q?clMpm+fIoxeRRcdLDFonU3aHW3UHt/nDV6zrvKT2++K9n6dscc+QyHuI?= =?Windows-1252?Q?ta721CydlegSkHhtlXNua/WUGl+gOrlV/m+vHsRfx36puVGq72v2pqVI?= =?Windows-1252?Q?/V9bL9DLrQsYl1MFBj5Xtbjfj5+mbF5gmBzGxgPmtpS+IZobKDQpdIK1?= =?Windows-1252?Q?mpMfcxCcVQjnLusOIvp9zKuSMaNJK71GZpzKKnLOCl+9aTcIXmkB2sOb?= =?Windows-1252?Q?xPiiv1RWtva2TSBIqphj12kYE80McrqqaZng+8USxSCwUJienaDaM9Hj?= =?Windows-1252?Q?9NuIDFRErlTR4MxK5vC8RFkSlfMam+5JBzotXBfOq+g05Q3ZreUv+Tlx?= =?Windows-1252?Q?KkCYjfvcfwbsQ4cYbOlatnR6asj0BlEJUusYjH95QFadMKiVLlH0+jiq?= =?Windows-1252?Q?WtER5Zga9w5a3EfPSlAPKEs4zEOKsxyw+sb4Vh6OsX+ZUQvKlIUCOB7q?= =?Windows-1252?Q?6qz8venrSWUvFvsgApTF7eyO/hYoYw8y77y3KAsSu0AJJ2Yrp24nvAJa?= =?Windows-1252?Q?tZrHcoG5cBZn+E99+4sXME4+op2GTO5p5GEuE2AcjS4Tg8GR4k3KFz/V?= =?Windows-1252?Q?fYVqava/1jrQY18kP3mzei7T2d3gNgJrkClmtTbpmvZfpEfCpc9ehr14?= =?Windows-1252?Q?AebeR+YQwvvwMgtUCfhabkwyelasQTLuDh+zVkJHwL6iTjsnzFVjdF5r?= =?Windows-1252?Q?NN0Kbw1qxvxP3F7a3Jp/9R8oI/C1Vre0OaU3UvPF5LELSdHN4dPB5iAM?= =?Windows-1252?Q?P0ZgTKEzJM/I6WHExFoiMY2U8NnRx67Dife3eFhEwzlUXwTj2Yo7JNjJ?= =?Windows-1252?Q?7WOPwqhMzczWGkJM54hwTFtXzrEnYogjsqfX5RdCGL5rH0EMh0wCnS06?= =?Windows-1252?Q?ELN6C5tH6QtaLOTOP6UgZlAoc6cDGpr5sv2xFKc1OvhGs5uWxECs02tS?= =?Windows-1252?Q?UL8gnX4VyHrPu2BhWu0918+j/dPCeNb/8RT/PC7gVuswhNpsVSFMEHwt?= =?Windows-1252?Q?DSKcczJSfLOEb4Yj15/uSTnd/DnRhRcPf+8iEq/7h6uMHyDr3RDOBA1b?= =?Windows-1252?Q?r85mTM+V1ap0XtJCfYgURLoqEPgoDNsMdoP4npIvdmub/D4ihFXIhAg+?= =?Windows-1252?Q?bkubEsYInj9xMfg1PajL55dsIC8u7QVsTb7GTySppEjCzECoONMnOOHH?= =?Windows-1252?Q?KZ0ltwo5nycqwvwxIY5wO3WRvPoLCye6ETLxcoKvr4irulpSAvBLKudf?= =?Windows-1252?Q?tYjtzmEJvESBCDjCQh4+06H/AuWRjuFYJBMzFpDmrc6jy+D7YEyXMOk1?= =?Windows-1252?Q?e2/m91nL4RyqZce0PZ+rSuRQ?= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL0PR06MB5009.namprd06.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(376014)(366016)(38070700021)(8096899003);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?Windows-1252?Q?I7y6ZzRZ5UYwideomGkMZFgkdjP1zZOz5uYXZoGzn8dFFXwbcmHmmSK1?= =?Windows-1252?Q?PS5ilGaFg5tp22a9kOYZiEE1dx7KXpF4c+LP5lVdKywN5DqRJmbFZlgq?= =?Windows-1252?Q?kBNfKRWNGvV3k5LbWDFcRphSdiftwYmde3DJpBkVV9eZ4+bkDbcdHWrH?= =?Windows-1252?Q?vyPXsXn1PXE9jXi9U4y9xWl/g9QtJ/yn2+/4hXxDG+8UT0D60EjaW2HK?= =?Windows-1252?Q?mbnNj1E1EVQbSapeAngeugoi6k24BD+Qvw9xbp3wMFzuvMBrDpqSRDSo?= =?Windows-1252?Q?0WJxRSzzIerKgAJSOH29E1KXg3hsmTbqBN0xazE6Dpje7eNiwxmX0fG5?= =?Windows-1252?Q?04B0+/xRNbEsMVDv2KEA2dJTI/H38Nya1/ZwCdE2wARbCP441lkzVc1g?= =?Windows-1252?Q?ozj2GuvZxWzOukcEddTBOakPje7grLS8WG4D4Tpff4BUVvdeh6u7RPzh?= =?Windows-1252?Q?P+4Uk6IYeDPqwAmnTH4aV+MzqiY3jW9r1zaru17Jpx0epZeRymG0M+KV?= =?Windows-1252?Q?lDlTudxvN6l+vo2g0InZ7iFmPG9hbaLDNP8s+z3bHzLuGsplMXgBEO5J?= =?Windows-1252?Q?59vWjJuelZD/rsYMupg4+kMJsPhUSmwILo9qS/9bjT138tJL39uM7YFm?= =?Windows-1252?Q?kNituTR4xMaQSxTwDNCCQuVBu1xTbri3rdNMvAKh/h9XlP1YEvNrsuMr?= =?Windows-1252?Q?FM86RyQhnA6z9RbollrU7gnX9cPCfClwwcqqMrVVqhignrQfbBO+ANi2?= =?Windows-1252?Q?eteFrBeK9CFOlil9Yq8iBCaf9mYGoYOkSifJ4/M3FiWQwCY88XUPy1TR?= =?Windows-1252?Q?helRloU7vciEVIf35sKFaOdsVha4GppKpYoHTA/kIbb3iCUWnHJgprKP?= =?Windows-1252?Q?i+7tIrbgrUXYZfOVDkUqCP3/mTHYZrdFeCDDEeI9VcR2MoMsfzugrOga?= =?Windows-1252?Q?4viRhmtVJOPFF4VlZMvb7gUMbWOLd6o4iIBZMwj+2BAJrufcAKmQAS8W?= =?Windows-1252?Q?RbbM+ci63Uedc/gqAL7kXbYy5uX1XUOYF6wPOXzfaNRtpmdKtyaqLNzb?= =?Windows-1252?Q?aCYpn7MQO/di/0c751u4a7TltUfSybLNaaUMoSwB2bbEOFiqE3nYKu4s?= =?Windows-1252?Q?c7EdRkzmSf9V6egIHpgfbyI50H6flagQsr5hc/rQzI1kX0++8vEvqYvV?= =?Windows-1252?Q?YN6c531z9IHmSdT+w7HEor9c1PN4CWMoXime9VmwHZ6VAKVSko7vEZya?= =?Windows-1252?Q?6JRy/qdoFsUJbSCmf8BOygNxARCpfgp/7gpZxECFW/7M4TL2kfszABtP?= =?Windows-1252?Q?dAEG5BNFWJgDIMRhx8GLnvmnCq5+KrPG7YfcYUcnHGjdsQ/IRfkTMTo9?= =?Windows-1252?Q?qiArjqXJFh2wy5ojC5oH2xKkT2lYKXVwdxYrrFmAVCoxDPG1MtcOHo+n?= =?Windows-1252?Q?ST2Al/q1IVfBMohBWcjxqOw+kP9uCV4W6Ck6XUycRLj4xB7OTLFn/DcK?= =?Windows-1252?Q?OM3FqeRKPtmF5+1fYNx8jjCOLaSWhDRQzhTCp6/8qVJ7GFxRu/mACkPj?= =?Windows-1252?Q?9OUjS2edGhjcyA/r/e98KJrn4E8iAe+ghxp6UuYM4zGT68nZzqRvwEk8?= =?Windows-1252?Q?vejOVY1G2U0+0PopXEnA1mSBheVYnJT3b3hmpkstNOoU0nwzSpm2naBG?= =?Windows-1252?Q?97E6Ozcjotod1qWtCe/rqJ5hZvENbKhbe5PLU1XsWgxlq/+4hb1nqCIj?= =?Windows-1252?Q?QzoZ2wWnxyGzRxVAYhiXBIa+T4gye80j0WJe/5fZTZDHD2VGgmjOE+dD?= =?Windows-1252?Q?WVAGE5Y9VRYm+B4Da0InlRl6s3M9u3vFGQnbg+IVQhn0lZSad2WhBs4A?= =?Windows-1252?Q?dTnE7VKvdsK4qE9cwC4AHwziVIcBWYiDqLE=3D?= Content-Type: multipart/alternative; boundary="_000_BL0PR06MB5009A7B383ACA34CD3603B058A82ABL0PR06MB5009namp_" MIME-Version: 1.0 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: qzttd7rCIHaAAGpW2e1BBP8jBTb2TJvvNZa1K88zzipCA/T2n8TzbEsrwPuTzmkUzc6It9e8K9vqOuHuVfsFtRrfU5UX7LMK3cGsyPkpvnaWWE+ysiSgUEKilmMW+fGVch043M4zeb56u7xMsCP52U2jWSzj+JInmeqHcRa7DyI5QHPVcM8vLUdvjd5UYd6lKZLqZwhoOdWEEGqFr59oCfzD/bRL5DO2MopmEzgKlfxeKPQv53XUiQActXSZMWU6RYyQ3S5xttjjyHUxGCqo9Wxv+LQ6/zQm+c0ORq6LucpOgTqoq8L5HD/ceb0Tz+2AqLlNS/wyXLTN77GRCD9l+AVOR1pXS0aEdSkA/DOkgOhtRNajwBXYiOeN9gzqxw0sLojygc/2VC3pi1Xvz8bIzxCem69m+7A3xCTg4ofYvGVtTpje8rpYQ1oE8A8Z8+Obg6MSe1NumkDuBw6btlgAnq4DebAeFPeMFTuMUGJBRGx0c94vYI7R+0yUB3I1H+R4K+/AzrGh45Jf7HCq5XczKW/X14Whdjtp5BNf+oMw2PBQymo/ue1eqlKP4ZJ0X7JRQsaVlLDc5u3aepq1dWv2NFGECYYC14aWi9BHEEnFd3OriG+AiVMRYPGL40z+dGjD X-OriginatorOrg: BestBuy.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BL0PR06MB5009.namprd06.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8d48defc-f507-476e-7703-08de4fc7dba5 X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Jan 2026 21:41:33.5081 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 135e8995-7d3b-4466-844b-a0d62ba5f495 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: xygD1eMsHRzQgSWePU2DlEv3BLK4F44ZubmcxGKd/tOiCCIDH3KjFt0CJrtPy+M+3xWOrNRFZ0+uOjazZuwopQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH7PR06MB10596 X-MS-Exchange-CrossPremises-AuthAs: Internal X-MS-Exchange-CrossPremises-AuthMechanism: 04 X-MS-Exchange-CrossPremises-AuthSource: BL0PR06MB5009.namprd06.prod.outlook.com X-MS-Exchange-CrossPremises-TransportTrafficType: Email X-MS-Exchange-CrossPremises-SCL: 1 X-MS-Exchange-CrossPremises-messagesource: StoreDriver X-MS-Exchange-CrossPremises-BCC: X-MS-Exchange-CrossPremises-originalclientipaddress: 168.94.238.42 X-MS-Exchange-CrossPremises-transporttraffictype: Email X-MS-Exchange-CrossPremises-antispam-scancontext: DIR:Originating;SFV:NSPM;SKIP:0; X-MS-Exchange-CrossPremises-processed-by-journaling: Journal Agent X-OrganizationHeadersPreserved: CH7PR06MB10596.namprd06.prod.outlook.com X-Authority-Analysis: v=2.4 cv=b5O/I9Gx c=1 sm=1 tr=0 ts=69617610 cx=c_pps a=pVIVKlmwkrZr9AIty3oQVA==:117 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=vUbySO9Y5rIA:10 a=YABJ2S72nEIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=We9uveLrYNULbZ6W_oUA:9 a=pILNOxqGKmIA:10 a=m_YdO7WZ5BtRv1h_:21 a=_W_S_7VecoQA:10 X-Proofpoint-ORIG-GUID: SSIlbySUNg4LLpTN9TV6dbQXPwb1gc0O X-Proofpoint-GUID: SSIlbySUNg4LLpTN9TV6dbQXPwb1gc0O X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTA5MDE3MSBTYWx0ZWRfX43MpXbYWh9Qh EfXNfO8HttKC9xqeyeD5OyUb+1ZeZVYZR8diVvt6eI+nikIN7BPtUb10lwmno++ZCtQVegVP9QO aTchingQ0AuRYtyjhVfSshxIgpDGBOTFurHXvRoO9NUodaFhi3J2fpGUMp1nZfbUzSbNpTApfqa 5M1uP2iv4DIJ+hIvmmKcPfMsmY+0b6r4GL7YjA/mn0Kr5SSIn1DpzKED9Fm5Keph/sYGDwkOAGD EiuMhmE1sL1q0y+3xQ2b4eaJJjPKWHRFHp5SR1hgcjsloP4ZkCKZPAbFtakqa8KyUJ+DBWCtA3t YUI5nx1YeLhSa/+uXJc0HMQEN9djVG81KK+Krl1Mqw6pxkZALZqbiUQ638JLRbtb5LnoHTNipBw p4cXsS8DRvzTcsCuV2NJzK2NFazmCFEqcXB4y8+k2XyDXC7apu6RRDR2BdiET0AKYil9owazJVv EgSAa3jXuPQLISr+rjw== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2026-01-09_06,2026-01-09_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 spamscore=0 malwarescore=0 bulkscore=0 clxscore=1011 lowpriorityscore=0 suspectscore=0 impostorscore=0 priorityscore=1501 adultscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2512120000 definitions=main-2601090171 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --_000_BL0PR06MB5009A7B383ACA34CD3603B058A82ABL0PR06MB5009namp_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Aurora Postgres version 17.4 This application mainly runs 3 queries very frequently- around 12000 execu= tions per second across the 3. Normally the server handles this fine and a= t any given time there are between 0 - 10 active sessions running the queri= es. The 3 queries are tuned- doing 3-7 logical reads per execute and run = in < .1 millisecond. Periodically there will be a huge spike in active sessions up to 400+ sessi= ons all on CPU (it can be higher depending on how many app servers are in t= he mix at the time). This spike will last 2-5 seconds but it causes an inc= rease in app response time which is a problem. We snap info from pg_stat_activity (where state <> =91idle' ) every 1 secon= d for troubleshooting issues and a couple observations from that snap data = when this spike happens: 1. The app sessions will show as wait_event_type and wait_event as null which = I understand means they are on CPU. 2. Some of those sessions will not have queryid populated. But all the app se= ssions do is execute queries. 3. With aurora Postgres they store plan information in pg_stat_activity in pla= nid column. For some sessions the queryid will be populated but the planid= will not be populated. According to AWS this can happen if the session ha= s completed parsing the query but has not finished plan generation or sessi= ons are in the bind phase where parameter binding occurs before final plan = selection. With Aurora Postgres we do not have access to the server to look at the uni= x processes but we enabled enhanced monitoring in 1 second intervals which = does provide a little info on session processes. One thing that stood out = when the active session spike happens if I looked at the process list (it = only shows ~ 95 of the processes) I would see a bunch like this: "vss": 640900784, "name": "postgres: prcoffrcnsmruser prcoffrcnsmr 172.29.145.159= (53026) BIND", "tgid": 26247, "parentID": 584, "memoryUsedPc": 0.02, "cpuUsedPc": 0.06, "id": 26247, "rss": 30496, "vmlimit": =93unlimited" Where they had BIND next to the process info (vs others will show things li= ke idle or SELECT). In this case of the 95 processes shown it may have sho= wn 59 of them were idle, 2 were SELECT, 2 were PARSE, and 32 were BIND. So based on these processes showing BIND and pg_stat_activity either not sh= owing a queryid or planid if it did show a queryid it seems these sessions = may be getting hung up parsing the queries or specifically at the step it d= oes binding. Can anyone come up with any theories what may be going on here or is my des= cription/explanation above too vague? The app is using prepared statements= . The connection pool is HikariCP which does not support caching prepared = statements. The thing that throws me off is so far I=92m unable to see any= thing the app is doing to cause this. I don=92t see a spike in query activ= ity. I don=92t see a spike in new connections from the pools. I don=92t s= ee anything leading up to this other than this spike in sessions out of now= here. I don=92t see any resource issues on the server. It will run this s= ame load consistently throughout the day and the active session spike may h= appen 0 - 10 times in a day. And ideas would be appreciated. --_000_BL0PR06MB5009A7B383ACA34CD3603B058A82ABL0PR06MB5009namp_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable
Aurora Postgres version 17.4

This application mainly runs 3 queries very frequently-  around 12000 = executions per second across the 3.  Normally the server handles this = fine and at any given time there are between 0 - 10 active sessions running= the queries.  The 3 queries are tuned-  doing 3-7 logical reads per execute and run in < .1 millisecond.

Periodically there will be a huge spike in active sessions up to 400+ sessi= ons all on CPU (it can be higher depending on how many app servers are in t= he mix at the time).  This spike will last 2-5 seconds but it causes a= n increase in app response time which is a problem.  

We snap info from pg_stat_activity (where state <> =91idle' ) every 1= second for troubleshooting issues and a couple observations from that snap= data when this spike happens:

  1. The app sessions will show as wait_e= vent_type and wait_event as null which I understand means they are on CPU.<= /div>
  2. Some of those sessions will not have= queryid populated.  But all the app sessions do is execute queries.&n= bsp;
  3. With aurora Postgres they store plan= information in pg_stat_activity in planid column.  For some sessions = the queryid will be populated but the planid will not be populated.  A= ccording to AWS this can happen if the session has completed parsing the query but has not finished plan generation or se= ssions are in the bind phase where parameter binding occurs before final pl= an selection.

With Aurora Postgres we do not have access to the server to look at the uni= x processes but we enabled enhanced monitoring in 1 second intervals which = does provide a little info on session processes.  One thing that stood= out  when the active session spike happens if I looked at the process list (it only shows ~ 95 of the processes) I wo= uld see a bunch like this:

  "vss": 640900784,
            "name": "postgres:= prcoffrcnsmruser prcoffrcnsmr 172.29.145.159(53026) BIND",
            "tgid": 26247,
            "parentID": 584,
            "memoryUsedPc": 0.02,
            "cpuUsedPc": 0.06,
            "id": 26247,
            "rss": 30496,
            "vmlimit": =93unlimited= "

Where they had BIND next to the process info (vs others will show things li= ke idle or SELECT).  In this case of the 95 processes shown it may hav= e shown 59 of them were idle, 2 were SELECT, 2 were PARSE, and 32 were BIND= .

So based on these processes showing BIND and pg_stat_activity either not sh= owing a queryid or planid if it did show a queryid it seems these sessions = may be getting hung up parsing the queries or specifically at the step it d= oes binding. 

Can anyone come up with any t= heories what may be going on here or is my description/explanation above to= o vague?  The app is using prepared statements.  The connection pool is HikariCP which d= oes not support caching prepared statements.  The thing that throws me off is so far I=92m unable to see anything = the app is doing to cause this.  I don=92t see a spike in query activi= ty.  I don=92t see a spike in new connections from the pools.  I = don=92t see anything leading up to this other than this spike in sessions out of nowhere.  I don=92t see any resource issues on the= server.  It will run this same load consistently throughout the day a= nd the active session spike may happen 0 - 10 times in a day.

And ideas would be ap= preciated.

--_000_BL0PR06MB5009A7B383ACA34CD3603B058A82ABL0PR06MB5009namp_--