From: Tom Lane <tgl@sss.pgh.pa.us>
To: Tomas Vondra <tomas.vondra@2ndquadrant.com>
cc: Andres Freund <andres@anarazel.de>,
        "Todd A. Cook" <tcook@blackducksoftware.com>,
        Thomas Munro <thomas.munro@enterprisedb.com>,
        PostgreSQL Bugs <pgsql-bugs@postgresql.org>
Subject: Re: BUG #14932: SELECT DISTINCT val FROM table gets stuck in an
 infinite loop
In-reply-to: <7de4b2d6-b119-7d15-04d8-4c0fcb29cbc3@2ndquadrant.com>
References: <7a22f496-92b5-d199-984e-183965918304@blackducksoftware.com>
 <20171127221041.mezzysc4zcfsqkyc@alap3.anarazel.de>
 <CAEepm=3SC9zYLKhvtZbsnwf9vEpi96WCGz1rdGLofuzBau=fqA@mail.gmail.com>
 <17357.1511841783@sss.pgh.pa.us>
 <9cd29b81-7ba1-91a2-f491-9014f4ac87e6@blackducksoftware.com>
 <2a76784c-06cb-46ec-35be-9895d144b839@2ndquadrant.com>
 <219b37b2-d740-d04a-203a-49ae422028f7@blackducksoftware.com>
 <c15378b3-d4aa-a5c0-5538-dd52904dcbc6@2ndquadrant.com>
 <1773c302-2fb1-749f-8ccc-62b23617c5ac@blackducksoftware.com>
 <25629.1512580464@sss.pgh.pa.us>
 <20171206171923.dg7phzfh7nb4lou3@alap3.anarazel.de>
 <7de4b2d6-b119-7d15-04d8-4c0fcb29cbc3@2ndquadrant.com>
Comments: In-reply-to Tomas Vondra <tomas.vondra@2ndquadrant.com>
	message dated "Sun, 10 Dec 2017 23:09:42 +0100"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <28576.1515512208.1@sss.pgh.pa.us>
Date: Tue, 09 Jan 2018 10:36:48 -0500
Message-ID: <28577.1515512208@sss.pgh.pa.us>
Precedence: bulk

Tomas Vondra <tomas.vondra@2ndquadrant.com> writes:
> On 12/06/2017 06:19 PM, Andres Freund wrote:
>> On 2017-12-06 12:14:24 -0500, Tom Lane wrote:
>>> In any case, the hashtable code needs to not fall over in the
>>> presence of a lot of collisions, regardless of the exact reason
>>> for there being a lot.

>> Yes, we need to be more resilient about it. Working on a patch.

This seems to have fallen through a crack over Christmas holidays.
Let's see if we can't push it to a conclusion.

> [ Tomas presents some draft patches and performance analysis ]

> It seems only the "stop growth" has to be part of the solution, as it's
> included in any combination fixing all cases shared in this thread. I'd
> say (randomized hashing + stop growing) seems like the best option.

> FWIW I do agree the data sets shared in this thread are pretty extreme
> and it doesn't make much sense to slow the regular cases. I'll be
> perfectly happy if we stop the OOM, making those cases fast is a bonus.

Personally I'd say let's just adopt the "stop growing" patch and call it
good.  In particular, I'm nervous about the idea of backpatching your RH
patch because of the ABI hazard from changing struct TupleHashTableData.

But perhaps Andres has some different idea in mind ...

			regards, tom lane