Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1vWJ3L-00GoGp-1q for pgsql-general@arkaria.postgresql.org; Thu, 18 Dec 2025 18:48:44 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.96) (envelope-from ) id 1vWJ3K-003dCz-1k for pgsql-general@arkaria.postgresql.org; Thu, 18 Dec 2025 18:48:43 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1vWJ3K-003dCp-0e for pgsql-general@lists.postgresql.org; Thu, 18 Dec 2025 18:48:42 +0000 Received: from mail-qv1-xf36.google.com ([2607:f8b0:4864:20::f36]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96) (envelope-from ) id 1vWJ3J-001NRU-0i for pgsql-general@postgresql.org; Thu, 18 Dec 2025 18:48:41 +0000 Received: by mail-qv1-xf36.google.com with SMTP id 6a1803df08f44-8888a1c50e8so11569096d6.0 for ; Thu, 18 Dec 2025 10:48:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1766083720; x=1766688520; darn=postgresql.org; h=mime-version:message-id:date:user-agent:subject:to:from:from:to:cc :subject:date:message-id:reply-to; bh=KloN/h0mFUfVYa3JoL17JYjrDgBR5cT7kJ2ROAVOh3I=; b=m/+Dj0QDN0ousPZlo/mgBoIpM3AGkxCmvj1vF3UTtAlcae2fPo7coQmQXA9tPi7Uhh fsSjHv7r5R44c5gePURvRlkDNV/rUf+vYuDwpKSzX6z70x7Y7CUBnnSpbt13/ASm6qt7 DZ04Q79RadzFEVSOyWYdS0AsYbgv8PmJNIp5U6NfuFfc/TBHFt4dyD4pEN4dqjKhDRaW mlUDXSwLVCygG5PSS8F3jPEeU6lFGTAo/2GmsV6RZwd1wA+k2IzhivNGvYdkDLnrRUMJ UF6MycdIqgRkJRlF5aImS1DQ5eJ3+yHfef3gKi2mE3QDjvVgHlGOWfXD8IzCdiovenmJ X+tA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1766083720; x=1766688520; h=mime-version:message-id:date:user-agent:subject:to:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=KloN/h0mFUfVYa3JoL17JYjrDgBR5cT7kJ2ROAVOh3I=; b=EtEZMCH/HzR5ozuVentBwX6107mN6SRYFl3tGsgR8grjJyKk6JC5A1uTlsWl3wNV2E v+yXL4Z/+1PyDyIdY42myNECL5Qe9ydUP7vaRZa/tukU9kEV9ejFj8Im8INdHnPpxzM6 Dmde1gLicyisKLORR8/fOdC0N7YxMgXOLAifqN02zcU7mYa6IZY21AtBbpp5TCU+64b0 1OcNMvVSblEM14oP5jk98q77KEqYgle4WS4Sz3hlL2CCtr0ccKlg8T1k8nl+cOL+7hes Ozda4lJXwA3Vc/aTNRRQFN1yhEd9bKVShh9T8sh5XRNciq+WcEcu/NE5DgcsusxcImRA v9yw== X-Gm-Message-State: AOJu0YyBY1zP8Y4GEB2CGj3oXPn6ElH+T4PvFKJMzYNymvDEKnxTWI32 5VpG9xSRtOpCq/S/i8ZBuZ8ZFx1/1C4Qs1bApgp5/KArTpxjryPysJQ8qSCTUA== X-Gm-Gg: AY/fxX5Cun8FizheHPQCVeWKe+f2RPO9bVVaOe8HxAHNz1RUTZ5npTM7vLKOtltC0FK mmHdSVUy3meYsmwiYf6OJoSoJNo/MF1458k736y9TOy6l7e9UhWqtNXUb8wxTPNKqHgBpviqVMK PQuxVJxL4lXlfDUOLsEJkr1MaNNNBU6Pus9AuMXiWpcwHdfgMBFFjpKxs900JlehfXhucWDOjLW RqVmB9jhVEoSoiZku3EoveszdWNFupnTRlUxzqWIVxwXOBlWYLgmfgKz2lcyZHqAzd+88ZoZur2 abDxHpT9mU2n4j6fNYpAVcR8GXz6CjIChWmbBqAJ9ugGiiz+zWpzQjoWd31AbaS5UqSYBDOmtkW Xxaz2RY4OHWb8JWiudLcPBGAH1YAxwxfC42bQ5JSZ+jBHl4kSK8Ipxf2bzehDHdD8uK1Vs7r57H IO9hu8B0c3A/I4v4RlLsj64pMgSzBrVA== X-Google-Smtp-Source: AGHT+IEAjbzv078nSYgs4WT5ppQxC09zSauw5hkwqhAq+pZ7r/tG/Dsux5TJzvt+l+i0cMLR2KHvFQ== X-Received: by 2002:a05:6214:3110:b0:88a:568b:f4af with SMTP id 6a1803df08f44-88d828162edmr11676826d6.23.1766083719813; Thu, 18 Dec 2025 10:48:39 -0800 (PST) Received: from mp-st-nix-fw.gmail.com ([2607:fb90:df0e:4472:ed1:12b6:1399:f573]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-88d996b69c4sm1760716d6.36.2025.12.18.10.48.39 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 Dec 2025 10:48:39 -0800 (PST) From: Matthew Planchard To: pgsql-general@postgresql.org Subject: Dealing with SeqScans when Time-based Partitions Cut Over User-Agent: mu4e 1.12.9; emacs 30.2 Date: Thu, 18 Dec 2025 13:48:37 -0500 Message-ID: <87bjjv1v96.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk In a table with high insert frequency (~1.5k rows/s) and high query frequency (~1k queries/s), partitioned by record creation time, we have observed the following behavior: * When the current time crosses a partition boundary, all new records are written to the new partition, which was previously empty, as expected * Because the planner's latest knowledge of the partition was based on its state prior to the cutover, it assumes the partition is empty and creates plans that use sequential scans * The table accumulates tens to hundreds of thousands of rows, and the sequentail scans start to use nearly 100% of available database CPU * Eventually the planner updates thee stats and all is well, but the cycle repeats the next time the partitions cut over. We have tried setting up a cron job that runs ANALYZE on the most recent partition of the table every 15 seconds at the start of the hour, and while this does help in reducing the magnitude and duration of the problem, it is insufficient to fully resolve it (our engineers are still getting daily pages for high DB CPU utilization). We have considered maintaining a separate connection pool with connections that have `enable_seqscan` set to `off`, and updating the application to use that pool for these queries, but I was hoping the community might have some better suggestions. - Matthew Planchard