MIME-Version: 1.0
References: <aWTVEycKj7Qh/SXH@ip-10-97-1-34.eu-west-3.compute.internal>
 <CAA5RZ0vgN8nf7yPrk_uNt-ABFtRnRuz7-9CM=qSzUSwj=EEUKw@mail.gmail.com> <aWjbBd5z+LJNWX1l@ip-10-97-1-34.eu-west-3.compute.internal>
In-Reply-To: <aWjbBd5z+LJNWX1l@ip-10-97-1-34.eu-west-3.compute.internal>
From: Sami Imseih <samimseih@gmail.com>
Date: Thu, 15 Jan 2026 11:25:18 -0600
Message-ID: <CAA5RZ0t6j0VYuUpxZ8JLq-ERoUriZ0rK=+8PCUtjRirmSmCx7A@mail.gmail.com>
Subject: Re: Flush some statistics within running transactions
To: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Cc: pgsql-hackers@lists.postgresql.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://www.postgresql.org/message-id/CAA5RZ0t6j0VYuUpxZ8JLq-ERoUriZ0rK%3D%2B8PCUtjRirmSmCx7A%40mail.gmail.com>
Precedence: bulk

> > > The 1 second flush interval is currently hardcoded but we could imagi=
ne increase
> > > it or make it configurable.
> >
> > Someone may want to turn this off as well. I think a GUC will be needed=
.
>
> I gave this more thoughts and I wonder if this should be configurable at =
all.
> I mean, we don't do it for PGSTAT_MIN_INTERVAL, PGSTAT_MAX_INTERVAL and
> PGSTAT_IDLE_INTERVAL. We could imagine make it configurable if it produce=
s
> noticeable performance impact but that's not what I observed.

Is there a reason we need a new constant (PGSTAT_ANYTIME_FLUSH_INTERVAL)
for anytime flushes and can't rely on the existing PGSTAT_MIN_INTERVAL?

Also, How did you benchmark? I am less concerned about long running
transactions,
background processes and more about short/high concurrency transactions see=
ing
additional overhead due to additional flushing. Is that latter a concern?

> > > stats: numscans, tuples_returned, tuples_fetched, blocks_fetched,
> > > blocks_hit
> >
> > I=E2=80=99m concerned that fields being temporarily out of sync might i=
mpact monitoring
> > calculations, if the formula is dealing with fields that have
> > different flush strategies.
>
> That's a good point. Maybe we should document the fields flush strategy?

Yeah, we will need to document this.

> > That said, minor discrepancies are usually tolerable for monitoring
> > data analysis.
> >
> > For the numscans, should we not also update the scan timestamp?
>
> The problem is that we could not call GetCurrentTransactionStopTimestamp(=
), so
> we would need to call GetCurrentTimestamp() instead. I'm not sure that ca=
lling
> GetCurrentTimestamp() every second would be a real issue though, and if i=
t is
> maybe we could increase this 1s value.

> That said I agree that having seq_scan being updated and not last_seq_sca=
n is not
> that great.

with v3 ,  I checked by running seq scans in a long running transaction,
and I observed both for these values being updated at the same time. I thin=
k
this is OK.

# pgstat_relation_flush_anytime_cb
```
tabentry->numscans +=3D lstats->counts.numscans;
if (lstats->counts.numscans)
{
TimestampTz t =3D GetCurrentTimestamp();

if (t > tabentry->lastscan)
tabentry->lastscan =3D t;
}
```
and

# pgstat_relation_flush_cb
```
if (lstats->counts.numscans)
{
TimestampTz t =3D GetCurrentTransactionStopTimestamp();

if (t > tabentry->lastscan)
tabentry->lastscan =3D t;
}
```

--
Sami Imseih
Amazon Web Services (AWS)