pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Tom Lane <tgl@sss.pgh.pa.us>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: Avoid overflow hazard when clamping group counts to "long int".
Date: Sat, 21 May 2022 17:14:03 +0000
Message-ID: <E1nsSfy-000T3n-Us@gemulon.postgresql.org> (raw)

Avoid overflow hazard when clamping group counts to "long int".

Several places in the planner tried to clamp a double value to fit
in a "long" by doing
        (long) Min(x, (double) LONG_MAX);
This is subtly incorrect, because it casts LONG_MAX to double and
potentially back again.  If long is 64 bits then the double value
is inexact, and the platform might round it up to LONG_MAX+1
resulting in an overflow and an undesirably negative output.

While it's not hard to rewrite the expression into a safe form,
let's put it into a common function to reduce the risk of someone
doing it wrong in future.

In principle this is a bug fix, but since the problem could only
manifest with group count estimates exceeding 2^63, it seems unlikely
that anyone has actually hit this or will do so anytime soon.  We're
fixing it mainly to satisfy fuzzer-type tools.  That being the case,
a HEAD-only fix seems sufficient.

Andrey Lepikhov

Discussion: https://postgr.es/m/ebbc2efb-7ef9-bf2f-1ada-d6ec48f70e58@postgrespro.ru

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/a916cb9d5a89804998dd4e7fd7bbb27cb5a7abc8

Modified Files
--------------
src/backend/executor/nodeSubplan.c      |  4 ++--
src/backend/optimizer/path/costsize.c   | 27 +++++++++++++++++++++++++++
src/backend/optimizer/plan/createplan.c |  7 +++----
src/include/optimizer/optimizer.h       |  1 +
4 files changed, 33 insertions(+), 6 deletions(-)



Message-ID: <E1nsSfy-000T3n-Us@gemulon.postgresql.org>
Permalink:  ../E1nsSfy-000T3n-Us@gemulon.postgresql.org/
Also on:    postgresql.org/message-id/E1nsSfy-000T3n-Us@gemulon.postgresql.org

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-committers@postgresql.org
  Cc: tgl@sss.pgh.pa.us
  Subject: Re: pgsql: Avoid overflow hazard when clamping group counts to "long int".
  In-Reply-To: <E1nsSfy-000T3n-Us@gemulon.postgresql.org>
  References: <E1nsSfy-000T3n-Us@gemulon.postgresql.org>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox