From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Thomas Munro <thomas.munro@enterprisedb.com>
To: Andres Freund <andres@anarazel.de>
Cc: Todd A. Cook <tcook@blackducksoftware.com>
Cc: PostgreSQL Bugs <pgsql-bugs@postgresql.org>
Subject: Re: BUG #14932: SELECT DISTINCT val FROM table gets stuck in an infinite loop
Date: Tue, 28 Nov 2017 14:08:10 +0100
Message-ID: <ef6e817b-4a60-cb70-5375-eaf71a82372c@2ndquadrant.com> (raw)
In-Reply-To: <CAEepm=3SC9zYLKhvtZbsnwf9vEpi96WCGz1rdGLofuzBau=fqA@mail.gmail.com>
References: <20171127185700.1470.20362@wrigleys.postgresql.org>
<fd924fd9-8f27-d945-04a9-57b503896107@2ndquadrant.com>
<7a22f496-92b5-d199-984e-183965918304@blackducksoftware.com>
<20171127221041.mezzysc4zcfsqkyc@alap3.anarazel.de>
<CAEepm=3SC9zYLKhvtZbsnwf9vEpi96WCGz1rdGLofuzBau=fqA@mail.gmail.com>
On 11/28/2017 03:20 AM, Thomas Munro wrote:
> On Tue, Nov 28, 2017 at 11:10 AM, Andres Freund <andres@anarazel.de> wrote:
>> On 2017-11-27 15:59:37 -0500, Todd A. Cook wrote:
>>> COPY reproducer (val) FROM stdin;
>>> 2976219712004784288
>>> -6429122065899879392
>>> -7471109962990387136
>>> -7471109962990387136
>>> -2895470491222113184
>>> -4083509061952565472
>>> 1019481548263425664
>>> 4639248884787347648
>>> -6999443831165647744
>>> -4199917803455020480
>>> -4110530183001439680
>>
>> How are these values generated? They awfully look like hash values
>> (~same lenght, full numerical range)...
>
> When SH_INSERT tries to insert that final extra value, insertdist
> keeps exceeding SH_GROW_MAX_DIB (25) no matter how many times we
> double the size (at least until my computer gives up, somewhere around
> 11 doublings and 75GB of virtual memory). If you set SH_GROW_MAX_DIB
> to 26 then it succeeds, but I guess some other attack could be crafted
> for that. What is the theory behind this parameter?
>
Yeah, I came to the same hypothesis yesterday, but I see I failed to
keep pgsql-bugs on CC :-(
FWIW I believe the last doubling (from 2147483648 to 2*2147483648) is
doomed to fail due to the sizemask=0 bug. I mean, if oldsize=2147483648,
then newsize=2*2147483648, which triggers this:
if (tb->size == SH_MAX_SIZE)
tb->sizemask = 0;
which would explain why the last grow did not complete even after 40
minutes (while the 1073741824->2147483648 took 15 seconds). Because with
sizemask=0 the SH_NEXT/SH_PREV/.. can only ever returns 0.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-bugs@postgresql.org
Cc: tomas.vondra@2ndquadrant.com, thomas.munro@enterprisedb.com, andres@anarazel.de, tcook@blackducksoftware.com
Subject: Re: BUG #14932: SELECT DISTINCT val FROM table gets stuck in an infinite loop
In-Reply-To: <ef6e817b-4a60-cb70-5375-eaf71a82372c@2ndquadrant.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox