pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Tom Lane <tgl@sss.pgh.pa.us>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: Be more wary about 32-bit integer overflow in pg_stat_statements
Date: Tue, 02 Aug 2022 22:05:56 +0000
Message-ID: <E1oJ01T-0002Jc-S1@gemulon.postgresql.org> (raw)

Be more wary about 32-bit integer overflow in pg_stat_statements.

We've heard a couple of reports of people having trouble with
multi-gigabyte-sized query-texts files.  It occurred to me that on
32-bit platforms, there could be an issue with integer overflow
of calculations associated with the total query text size.
Address that with several changes:

1. Limit pg_stat_statements.max to INT_MAX / 2 not INT_MAX.
The hashtable code will bound it to that anyway unless "long"
is 64 bits.  We still need overflow guards on its use, but
this helps.

2. Add a check to prevent extending the query-texts file to
more than MaxAllocHugeSize.  If it got that big, qtext_load_file
would certainly fail, so there's not much point in allowing it.
Without this, we'd need to consider whether extent, query_offset,
and related variables shouldn't be off_t not size_t.

3. Adjust the comparisons in need_gc_qtexts() to be done in 64-bit
arithmetic on all platforms.  It appears possible that under duress
those multiplications could overflow 32 bits, yielding a false
conclusion that we need to garbage-collect the texts file, which
could lead to repeatedly garbage-collecting after every hash table
insertion.

Per report from Bruno da Silva.  I'm not convinced that these
issues fully explain his problem; there may be some other bug that's
contributing to the query-texts file becoming so large in the first
place.  But it did get that big, so #2 is a reasonable defense,
and #3 could explain the reported performance difficulties.

(See also commit 8bbe4cbd9, which addressed some related bugs.
The second Discussion: link is the thread that led up to that.)

This issue is old, and is primarily a problem for old platforms,
so back-patch.

Discussion: https://postgr.es/m/CAB+Nuk93fL1Q9eLOCotvLP07g7RAv4vbdrkm0cVQohDVMpAb9A@mail.gmail.com
Discussion: https://postgr.es/m/5601D354.5000703@BlueTreble.com

Branch
------
REL_15_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/82ebc70d1c7fd9b301e15cec658696d28df01835

Modified Files
--------------
contrib/pg_stat_statements/pg_stat_statements.c | 26 +++++++++++++++++++++----
1 file changed, 22 insertions(+), 4 deletions(-)



view thread (7+ messages)  latest in thread

Message-ID: <E1oJ01T-0002Jc-S1@gemulon.postgresql.org>
Permalink:  ../E1oJ01T-0002Jc-S1@gemulon.postgresql.org/
Also on:    postgresql.org/message-id/E1oJ01T-0002Jc-S1@gemulon.postgresql.org

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-committers@postgresql.org
  Cc: tgl@sss.pgh.pa.us
  Subject: Re: pgsql: Be more wary about 32-bit integer overflow in pg_stat_statements
  In-Reply-To: <E1oJ01T-0002Jc-S1@gemulon.postgresql.org>
  References: <E1oJ01T-0002Jc-S1@gemulon.postgresql.org>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox