MIME-Version: 1.0
References: <87bjjv1v96.fsf@gmail.com> <CANzqJaAXhYwR6vHzm_DqnxmWk8QCYxEmhs_JeZ+w3F+rx09xSw@mail.gmail.com>
In-Reply-To: <CANzqJaAXhYwR6vHzm_DqnxmWk8QCYxEmhs_JeZ+w3F+rx09xSw@mail.gmail.com>
From: Matthew Planchard <msplanchard@gmail.com>
Date: Thu, 18 Dec 2025 14:55:12 -0500
Message-ID: <CAHncFmSsXwmLCKs-ySFWe=gjb=rSaSsVZQV6hJuDczhLixidqA@mail.gmail.com>
Subject: Re: Dealing with SeqScans when Time-based Partitions Cut Over
To: Ron Johnson <ronljohnsonjr@gmail.com>
Cc: pgsql-general <pgsql-general@postgresql.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://www.postgresql.org/message-id/CAHncFmSsXwmLCKs-ySFWe%3Dgjb%3DrSaSsVZQV6hJuDczhLixidqA%40mail.gmail.com>
Precedence: bulk

Thanks very much for the reply and the suggestions, a lot of good
stuff to look at.

> What's autovacuum_analyze_scale_factor set to?   The default 20% is prett=
y high.
> autovacuum_naptime might need to be dropped, too.

This is just whatever the default is, so I will look into updating
these settings.

> And maybe have the shell script that the cron job runs sleep only 5 secon=
ds in the ANALY loop.

Good suggestion. Our job runner will not run duplicate jobs if one is
still running, so I may even just have this run every second.

> How about just force seqscan off when the table is created?
> ALTER TABLE <table_partition> SET (enable_seqscan  =3D off);

I didn't know this could be set on the table level! Our partitions are
created by partman, but we have a nightly job that does maintenance
stuff that could go through and update this setting on future partitions
before they come into use. I'm always hesitant to mess
too much with the planner, but this might be the ticket.

On Thu, Dec 18, 2025 at 2:32=E2=80=AFPM Ron Johnson <ronljohnsonjr@gmail.co=
m> wrote:
>
> On Thu, Dec 18, 2025 at 1:48=E2=80=AFPM Matthew Planchard <msplanchard@gm=
ail.com> wrote:
>>
>>
>> In a table with high insert frequency (~1.5k rows/s) and high query
>> frequency (~1k queries/s), partitioned by record creation time, we have
>> observed the following behavior:
>>
>> * When the current time crosses a partition boundary, all new records
>>   are written to the new partition, which was previously empty, as
>>   expected
>>
>> * Because the planner's latest knowledge of the partition was based on
>>   its state prior to the cutover, it assumes the partition is empty and
>>   creates plans that use sequential scans
>>
>> * The table accumulates tens to hundreds of thousands of rows, and the
>>   sequentail scans start to use nearly 100% of available database CPU
>>
>> * Eventually the planner updates thee stats and all is well, but the
>>   cycle repeats the next time the partitions cut over.
>>
>> We have tried setting up a cron job that runs ANALYZE on the most recent
>> partition of the table every 15 seconds at the start of the hour, and
>> while this does help in reducing the magnitude and duration of the
>> problem, it is insufficient to fully resolve it (our engineers are still
>> getting daily pages for high DB CPU utilization).
>
>
> What's autovacuum_analyze_scale_factor set to?   The default 20% is prett=
y high.
> autovacuum_naptime might need to be dropped, too.
>
> And maybe have the shell script that the cron job runs sleep only 5 secon=
ds in the ANALY loop.
>
>>
>> We have considered maintaining a separate connection pool with
>> connections that have `enable_seqscan` set to `off`, and updating the
>> application to use that pool for these queries, but I was hoping the
>> community might have some better suggestions.
>
>
> How about just force seqscan off when the table is created?
> ALTER TABLE <table_partition> SET (enable_seqscan  =3D off);
>
> --
> Death to <Redacted>, and butter sauce.
> Don't boil me, I'm still alive.
> <Redacted> lobster!