Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1vWJ6x-00Goz2-2o for pgsql-general@arkaria.postgresql.org; Thu, 18 Dec 2025 18:52:28 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.96) (envelope-from ) id 1vWJ6w-003g1m-2a for pgsql-general@arkaria.postgresql.org; Thu, 18 Dec 2025 18:52:27 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1vWJ6w-003g1d-1V for pgsql-general@lists.postgresql.org; Thu, 18 Dec 2025 18:52:27 +0000 Received: from mail-qk1-x72d.google.com ([2607:f8b0:4864:20::72d]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96) (envelope-from ) id 1vWJ6v-001NT0-2K for pgsql-general@lists.postgresql.org; Thu, 18 Dec 2025 18:52:26 +0000 Received: by mail-qk1-x72d.google.com with SMTP id af79cd13be357-8b25dd7ab33so72612285a.1 for ; Thu, 18 Dec 2025 10:52:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1766083945; x=1766688745; darn=lists.postgresql.org; h=mime-version:message-id:date:user-agent:subject:to:from:from:to:cc :subject:date:message-id:reply-to; bh=KloN/h0mFUfVYa3JoL17JYjrDgBR5cT7kJ2ROAVOh3I=; b=KYvdw4Tokz1wYCPIn7VLWsj3bvG7AJB86RFWo6Jd7FousFKYwiJnr7OSSEkmSO8xQO aHbxROm5zZF8KKhdvql5/hdB6SPqLLaEBpDTjxrKrJGBtF8NBOUDUj5GFZs3BFS9k4/q UVP3JiBuDczm28q1S+cIzy8UHPGC/JZCINyZFwin4qLshx1htIBMpkRnFiiV72Zm2HO3 FJaLgGisKtvCAv1PgS+URqrHeaDhNLwwTVNceS0mFPBDIrlaZFBIYXsBfbM7l+libKOj UL7NJzQ6hcWJpo/+NXLwr8ijgDYh4hM2AybbWap3Gv6K71/+zg+Vfo/sJFWvyqTFxr8y LE5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1766083945; x=1766688745; h=mime-version:message-id:date:user-agent:subject:to:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=KloN/h0mFUfVYa3JoL17JYjrDgBR5cT7kJ2ROAVOh3I=; b=IZQqk7UyEaKxKh3XzzURTkWnOinFxmFjsPfnmm6PoOuZ1v7oYfdChiQMZDuZyNYewO esSCb2djdYVIGM61aIElckVPghBY9UqytKaBEt30rrlBlUIG/yXtyRc+z4Dj0QFy9S+y Sf96bC1Xv1rfoGp9xRhgjhzxOAptqNlcVGYeU37L7hGaSwN8qAiaD/JkkxKuwtZ52UG9 Fm76MXY+sqYLzNTUiH/GvCel+oTIYteDXtSLAjjxSvFrEWRCDUXtJw//bw6wP/XPQFM6 9eIIh4H3AQIWS20ZeTHRW7jmV1PpN0Y1aDB1R+paW10NvuHT3lauOy6dJFqnE5jbxNXZ tWOQ== X-Gm-Message-State: AOJu0YyBOY0mEkBHFDIkNiRAzW3sRaWvB5cZt8KGJn7LpVHhiA9dYf+U p7JByn7tq/IlrBosbQo6iHJzAO2MYRbS8C2EyVoiVXdNYc0pDw8ujL6mPVUFfw== X-Gm-Gg: AY/fxX7Tysh7DjQGwlMuhRhHDD0ea7mshwG4Nncb+MP5Mm/XJCFlmnWE7afi8yJJUHs JOUtU4X1ZAgGG7mUBAhJ3DHXhItiAZhmxr3dvF3uTDPTMt+K4GrAAS7DoFJtRMX4MXuIj2whuYj mA3QU10H2QyryaV3zGf2BjyJq9PBFGbc23VgV2GARvmr2aXSXmGZEnN70IzRjMsfCfc7B14oYzt /5IfkTCKjcupeAOU23HqtRDsKk533qZZRBgo4U2DG2hwf1EFzco9Y1f9+FK3R+DiV+PBTuupF3Y a5YwMBdwHE8r3hiGSrYRAEMzNHPRgClghVeJ7/rgW4yDzfIRsvYF1f0J8t7mwgSXzr2tnsIKKrj iaJRa3VdFMkCZFVzSm7g6gZQ4uhKM2WfceI44eqi0YGBZK7JB3eJTMQ+uda9O8egczV66FPSxAo xcYvwNIBMrHbIigcBdWzRhFOSp/PAJTg== X-Google-Smtp-Source: AGHT+IGseWmAxHqvDqI/Afq3RTx1UvXaU5ukKc1m0q7pBTC9xWh+F6HZ9VlWd9V4oev/QvC6WAAgdw== X-Received: by 2002:a05:622a:1195:b0:4ed:e337:2e68 with SMTP id d75a77b69052e-4f4abdd152emr3402681cf.81.1766083944478; Thu, 18 Dec 2025 10:52:24 -0800 (PST) Received: from mp-st-nix-fw.gmail.com ([2607:fb90:df0e:4472:ed1:12b6:1399:f573]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-4f4ac54add6sm105091cf.2.2025.12.18.10.52.23 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Dec 2025 10:52:24 -0800 (PST) From: Matthew Planchard To: pgsql-general@lists.postgresql.org Subject: Dealing with SeqScans when Time-based Partitions Cut Over User-Agent: mu4e 1.12.9; emacs 30.2 Date: Thu, 18 Dec 2025 13:52:22 -0500 Message-ID: <875xa31v2x.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk In a table with high insert frequency (~1.5k rows/s) and high query frequency (~1k queries/s), partitioned by record creation time, we have observed the following behavior: * When the current time crosses a partition boundary, all new records are written to the new partition, which was previously empty, as expected * Because the planner's latest knowledge of the partition was based on its state prior to the cutover, it assumes the partition is empty and creates plans that use sequential scans * The table accumulates tens to hundreds of thousands of rows, and the sequentail scans start to use nearly 100% of available database CPU * Eventually the planner updates thee stats and all is well, but the cycle repeats the next time the partitions cut over. We have tried setting up a cron job that runs ANALYZE on the most recent partition of the table every 15 seconds at the start of the hour, and while this does help in reducing the magnitude and duration of the problem, it is insufficient to fully resolve it (our engineers are still getting daily pages for high DB CPU utilization). We have considered maintaining a separate connection pool with connections that have `enable_seqscan` set to `off`, and updating the application to use that pool for these queries, but I was hoping the community might have some better suggestions. - Matthew Planchard