pg.ddx.io  pgsql-bugs@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Cc: Andres Freund <andres@anarazel.de>
Cc: Todd A. Cook <tcook@blackducksoftware.com>
Cc: Thomas Munro <thomas.munro@enterprisedb.com>
Cc: PostgreSQL Bugs <pgsql-bugs@postgresql.org>
Subject: Re: BUG #14932: SELECT DISTINCT val FROM table gets stuck in an infinite loop
Date: Tue, 09 Jan 2018 10:36:48 -0500
Message-ID: <28577.1515512208@sss.pgh.pa.us> (raw)
In-Reply-To: <7de4b2d6-b119-7d15-04d8-4c0fcb29cbc3@2ndquadrant.com>
References: <7a22f496-92b5-d199-984e-183965918304@blackducksoftware.com>
	<20171127221041.mezzysc4zcfsqkyc@alap3.anarazel.de>
	<CAEepm=3SC9zYLKhvtZbsnwf9vEpi96WCGz1rdGLofuzBau=fqA@mail.gmail.com>
	<17357.1511841783@sss.pgh.pa.us>
	<9cd29b81-7ba1-91a2-f491-9014f4ac87e6@blackducksoftware.com>
	<2a76784c-06cb-46ec-35be-9895d144b839@2ndquadrant.com>
	<219b37b2-d740-d04a-203a-49ae422028f7@blackducksoftware.com>
	<c15378b3-d4aa-a5c0-5538-dd52904dcbc6@2ndquadrant.com>
	<1773c302-2fb1-749f-8ccc-62b23617c5ac@blackducksoftware.com>
	<25629.1512580464@sss.pgh.pa.us>
	<20171206171923.dg7phzfh7nb4lou3@alap3.anarazel.de>
	<7de4b2d6-b119-7d15-04d8-4c0fcb29cbc3@2ndquadrant.com>

Tomas Vondra <tomas.vondra@2ndquadrant.com> writes:
> On 12/06/2017 06:19 PM, Andres Freund wrote:
>> On 2017-12-06 12:14:24 -0500, Tom Lane wrote:
>>> In any case, the hashtable code needs to not fall over in the
>>> presence of a lot of collisions, regardless of the exact reason
>>> for there being a lot.

>> Yes, we need to be more resilient about it. Working on a patch.

This seems to have fallen through a crack over Christmas holidays.
Let's see if we can't push it to a conclusion.

> [ Tomas presents some draft patches and performance analysis ]

> It seems only the "stop growth" has to be part of the solution, as it's
> included in any combination fixing all cases shared in this thread. I'd
> say (randomized hashing + stop growing) seems like the best option.

> FWIW I do agree the data sets shared in this thread are pretty extreme
> and it doesn't make much sense to slow the regular cases. I'll be
> perfectly happy if we stop the OOM, making those cases fast is a bonus.

Personally I'd say let's just adopt the "stop growing" patch and call it
good.  In particular, I'm nervous about the idea of backpatching your RH
patch because of the ABI hazard from changing struct TupleHashTableData.

But perhaps Andres has some different idea in mind ...

			regards, tom lane




view thread (70+ messages)  latest in thread

Message-ID: <28577.1515512208@sss.pgh.pa.us>
Permalink:  ../28577.1515512208@sss.pgh.pa.us/
Also on:    postgresql.org/message-id/28577.1515512208@sss.pgh.pa.us

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-bugs@postgresql.org
  Cc: tgl@sss.pgh.pa.us, tomas.vondra@2ndquadrant.com, andres@anarazel.de, tcook@blackducksoftware.com, thomas.munro@enterprisedb.com
  Subject: Re: BUG #14932: SELECT DISTINCT val FROM table gets stuck in an infinite loop
  In-Reply-To: <28577.1515512208@sss.pgh.pa.us>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox