Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eMf0M-00012B-LS for pgsql-bugs@arkaria.postgresql.org; Wed, 06 Dec 2017 19:05:14 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eMf0L-0006Fk-T9 for pgsql-bugs@arkaria.postgresql.org; Wed, 06 Dec 2017 19:05:13 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1eMf0L-0006FS-NT for pgsql-bugs@lists.postgresql.org; Wed, 06 Dec 2017 19:05:13 +0000 Received: from mail-sn1nam02on0084.outbound.protection.outlook.com ([104.47.36.84] helo=NAM02-SN1-obe.outbound.protection.outlook.com) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.89) (envelope-from ) id 1eMf0F-0005Ge-Pz for pgsql-bugs@postgresql.org; Wed, 06 Dec 2017 19:05:12 +0000 Received: from MWHPR10CA0056.namprd10.prod.outlook.com (10.169.238.18) by BN6PR10MB1875.namprd10.prod.outlook.com (10.175.97.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.282.5; Wed, 6 Dec 2017 19:05:01 +0000 Received: from CY1NAM02FT061.eop-nam02.prod.protection.outlook.com (2a01:111:f400:7e45::207) by MWHPR10CA0056.outlook.office365.com (2603:10b6:300:2c::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.302.9 via Frontend Transport; Wed, 6 Dec 2017 19:05:01 +0000 Authentication-Results: spf=permerror (sender IP is 192.231.134.1) smtp.mailfrom=blackducksoftware.com; 2ndquadrant.com; dkim=none (message not signed) header.d=none;2ndquadrant.com; dmarc=fail action=none header.from=blackducksoftware.com; Received-SPF: PermError (protection.outlook.com: domain of blackducksoftware.com used an invalid SPF mechanism) Received: from ops-mail01.dc2.lan (192.231.134.1) by CY1NAM02FT061.mail.protection.outlook.com (10.152.75.30) with Microsoft SMTP Server id 15.20.282.5 via Frontend Transport; Wed, 6 Dec 2017 19:04:56 +0000 Received: from bds01237.bds-ad.lc (unknown [10.1.6.127]) by ops-mail01.dc2.lan (Postfix) with ESMTPS id 5AAA6434E8; Wed, 6 Dec 2017 14:04:56 -0500 (EST) Subject: Re: BUG #14932: SELECT DISTINCT val FROM table gets stuck in an infinite loop To: Andres Freund , Tom Lane CC: Tomas Vondra , Thomas Munro , PostgreSQL Bugs References: <7a22f496-92b5-d199-984e-183965918304@blackducksoftware.com> <20171127221041.mezzysc4zcfsqkyc@alap3.anarazel.de> <17357.1511841783@sss.pgh.pa.us> <9cd29b81-7ba1-91a2-f491-9014f4ac87e6@blackducksoftware.com> <2a76784c-06cb-46ec-35be-9895d144b839@2ndquadrant.com> <219b37b2-d740-d04a-203a-49ae422028f7@blackducksoftware.com> <1773c302-2fb1-749f-8ccc-62b23617c5ac@blackducksoftware.com> <25629.1512580464@sss.pgh.pa.us> <20171206171923.dg7phzfh7nb4lou3@alap3.anarazel.de> From: "Todd A. Cook" Message-ID: <359db34e-2bb1-8bee-4709-2b23ac7b22bd@blackducksoftware.com> Date: Wed, 6 Dec 2017 14:04:56 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: <20171206171923.dg7phzfh7nb4lou3@alap3.anarazel.de> Content-Type: text/plain; charset="utf-8"; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-EOPAttributedMessage: 0 X-Forefront-Antispam-Report: CIP:192.231.134.1;IPV:CAL;CTRY:;EFV:NLI;SFV:NSPM;SFS:(10009020)(346002)(376002)(2980300002)(448002)(24454002)(377424004)(199004)(189003)(93886005)(58126008)(6246003)(83506002)(316002)(97736004)(5660300001)(110136005)(106466001)(54906003)(65826007)(77096006)(53546010)(229853002)(65806001)(33646002)(2950100002)(67846002)(50466002)(65956001)(2486003)(230700001)(23676004)(85326001)(76176011)(478600001)(966005)(53936002)(64126003)(31696002)(47776003)(4326008)(305945005)(104016004)(36756003)(8936002)(31686004)(53416004)(6266002)(6306002)(81156014)(8676002)(81166006)(356003)(86362001)(2906002)(26826003)(217873001);DIR:OUT;SFP:1101;SCL:1;SRVR:BN6PR10MB1875;H:ops-mail01.dc2.lan;FPR:;SPF:PermError;PTR:ErrorRetry;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: 1;CY1NAM02FT061;1:7atUuY2n4uinKryT2PYQYRdWi8wWXLx91fN68nNzl57Smgkq6RUfpRo6y4M3iNxafjSXHpexpsuGyrzUlJ0y90jHOJ/YwJNFDozlk9kXpC/r1exgf6iSlSiKoU4h+PaF X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e3bc011c-c300-4b1b-6acc-08d53cdc3eef X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(4534020)(4602075)(4627115)(201703031133081)(201702281549075)(5600026)(4604075)(2017052603286);SRVR:BN6PR10MB1875; X-Microsoft-Exchange-Diagnostics: 1;BN6PR10MB1875;3:aEBxxk4xxKRzZ8XHtBG6DXTWejmwd+7dL6OA78LvIcBlHn54/UJp7umTmuGk22LE4h417s3xkDIyhDFFuJf4OslD+LWVuR+nVepAtH263eplFK4kO0cm2Lo3pU02MYBzkcp7QB+y/q9D/2FwWw1StGycslG76VSFKaeEf++IlS9zbu9vEIVODDBxBjiD+ws3T3Z8YBynC5lmxGXXb8ZzdHh0shhxVMlhCcNNa5BprmoIW/BwZFoIrt5RElMwZcAXuVzPU27rw8VDbXHVMgdoYCe2BXSxzyCpfkREb7ne8Ckss4lUjs/K95TboFtDv9fDGuMY3ICNoqgi2RXouGSSlg==;25:4csHw4bzwqUHwC600EQn6XNuqnSbbfJ78igg+KADst3Ms6mRkl9iRVVn2mgxlvPmvcX8nuJu7VZxTNNnymUZoxlnfY1bFbytFlLAoib4nQjxapwNCMSGQL0h53s4kjFDgFqRylnJPt077F4R4KXUeFxAI/JItYAm+bda9F04B13smP8AuipH1M1qkiwW0xIwhJINb/x/fRYXi8J3mvXp7kL8AacnmDJizqTcaeQW8kl75OJG1Ll25UW8aaR3qWuaW/84mmkA1lNzakJEbOdlqyANWtNy+HvxNgKWA0ciM/0BwyE3VgnCB2staPOjtQkh2UGcFBBJrWxc074beYSKOQ==;31:N1nXVukJT5xZsj+9OO2YQolf7AVO2LactMCtpjERy9/r/MLL0qr6JsgL7yPGHfIE6k4ugh21R/0vh9DoPnYi/iW2INpV+q7Rhn9OASsLwcfY2OkxSBk29FVFxeqq2fxzSroCyLJFOpniNe1FOFkT1rOduiL5nxExQUgHoWYxqUuO63gbcBvi2HmvwHOe2ZK/2Y4SGiTEqDtCBuzxEddD7UlmIKgGPX0Az7pwdbJREik= X-MS-TrafficTypeDiagnostic: BN6PR10MB1875: X-Microsoft-Exchange-Diagnostics: 1;BN6PR10MB1875;20:EKPTXJrY+wqa4pjm2VgAcMNY851CdabNh+1O0MyyzfrzB/xrkSX1uDTakhSmWhuHI5GcxEMd3Gpzu2A5QI+a5t/ivZFFLVV1xZ6NPzmuMGtu6roHbU47XXaDOrcqbMbxJ1H0nxId85RJkSBNGNJKuCM8MXTGe77/qJVImqUXs4QunQSJ3BqdP++PJgb+dOTKEBqIUGpsOR13BpGu+9t79oXC8oopmEIevvUFO3+sUPqAFny2XK3DQE5wOUtiq123H+nAxNKJ7tquAUbiXX/jrZi4Njelv/CsNexw5dIyMVXYcUccQlZWRiVRqFu+UIGISf/fkvmaNZ64vdBYkwBhOOixMnnMiz3LVZRKQ1KoM47MNzdqyMag2hvyQYlYOe7GYDcbB1yZPtJTU8eocq/8+/nqmurK+mLEATFhclmktQRYAuWOXBqkyBw+ym45xn/2tz6FWoefuh2L99grV6fPqpPe9aHWVdVrvP6EiAEPQTB/RkqfHwMWHn6bWtzqhgO9;4:ORg5s1izBw3CNpdpq7/ELBN/0F+bEY4oMeYgGNIbHachglvXyVEIAFRZwFnFg/AwJ5AtT02ur3UVKE9qi+6ZvOgJNXwkECzFb3b24e69qVAHHvuEVQAGYDZlHu9rNaK7BVkXHiYXA+aSl91s1hqLQEYGJtgQ7tXRuC8PNY37o+r1BEo6s6wz6yjmI40qMJnZMXAkHSbXF2HQ1UCu8/BE5DFqRq+I9qoNdzUZbnIDUn+qwVY8TgNxpS0LLkUxpxn894F5DxVYP9dP56nWt4BGLw== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040450)(2401047)(5005006)(8121501046)(3002001)(3231022)(93006095)(93001095)(10201501046)(6041248)(20161123555025)(20161123560025)(20161123564025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123558100)(20161123562025)(6072148)(201708071742011);SRVR:BN6PR10MB1875;BCL:0;PCL:0;RULEID:(100000803101)(100110400095);SRVR:BN6PR10MB1875; X-Forefront-PRVS: 05134F8B4F X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtCTjZQUjEwTUIxODc1OzIzOkVTWHovcEhrZHozSUhqS3Zqc2V4cDQzeFFm?= =?utf-8?B?bXFVNG5IU0xWTXJjVUszZFRLR3BQL2w0U2RLZ2FPaXBFTyt4dDJWT3FhZ0FX?= =?utf-8?B?ZnJrNDMva1M0UzFUQTYzNGNaZVh2WXJqd1hsUUNhL29GNGhkTW9ueWlqTGt6?= =?utf-8?B?bUNTWDdEUEc2UTdlNHRFRlJuUDZmN09uUXBxMHRvOWZRaVgrNFcvL0t4WmRr?= =?utf-8?B?RitEUHU1Q2RlRDBuNUI0a2trb0t0eUNweWdyb1NlYjcrMjRTNzExSjVWZDRh?= =?utf-8?B?c2ZPM0FXNVNCVWRyRU96UVNsYXQ1Z2tGNGpQTm1WOE5senQ3a0ZseitMSys1?= =?utf-8?B?cGJNMlRpR1VEcHpMUVdtZEtjN1NZZ0dCbW1tb1RjQnNTelBjbVBvVHJVS2Zs?= =?utf-8?B?a1lBcWw4cDJmK0JHTXFnQUtZVmdvOWh5ZmErajRyK3JpdFRKdkthRGJaVWNM?= =?utf-8?B?OE1JQ2lheURGRDViNi9aLy9BK0dBbFcxSnB2S1BNTGQ5WGdTMFJMM2Q3UWZX?= =?utf-8?B?aGdMQ0pGQVI5dFl5SHpqY05zNkV2RWJpbWdmYXBVWkVMQ2l3dE9OSDNmdGk4?= =?utf-8?B?RUl0QjFtZTNudHJtcUwwVG9rRGRDVFVYdTI0cnE3TDlWbDhwTFlzaWZRSE40?= =?utf-8?B?T1pqalV0ZXU3TnpBTnM1b1lkWU1TeStBWmdXWkpYbW4rbzdlZFBzNm1UaHpp?= =?utf-8?B?S3JQZEFaVnBkN2lPUTNWZVhXOGdYVGtWRm1Hbk10ems4WFBGRmViWThkdzVo?= =?utf-8?B?S1FLQXljK0xkYlBFdDlJWUV1NHh4WkRYWnJZWUJhZHIzeHJPbEk1a3V1U3JB?= =?utf-8?B?VTREVnZRMWpQeXhycjVBUFlvSXljbmRwTEpRMGVXbjFYRGZaY29tcmNGTFZO?= =?utf-8?B?b3FYWDFnWEF3UER0QytibCtDcW1lcDcxOFRGL1JFMDRIZlc4ZnNVaTFtbmFU?= =?utf-8?B?R0VoSVJYazRDNy92OWpyMkZodG1SaElOT2JWa0hvczFEeUtZWndVckQ3K2Ir?= =?utf-8?B?c08vSTlJTGhCSG5KNHBKSWZTM0lzeGJwcXE3Y3VOY1JkdjZKWGtlbUpSUklm?= =?utf-8?B?V05NMWtsL2VpZllHS2Y4VThtUGNoSmNCb1VVbGkzZ2c2bnpsSUtzOFVSOFpM?= =?utf-8?B?ZmxuTktzVmlybGVEZDVrTTZKR3FrdVV4UFBLYWFZeU1aaWZGMTh5UUg5c3JB?= =?utf-8?B?aG9ZOHdyQWRmYUNPRGY3MThSYXBDZzVEbFNJV1RVNzZ0ekU0RmFNUVFSbjJI?= =?utf-8?B?Q2prdEJKWWdwQU9BTkthK3QrL0pkMUpzY3ZCSU51ZXBVc1J6anhqOWExbkVH?= =?utf-8?B?cnpnWmQ3UGp0RTRaOTNTWXplUnZzaGpROEFiWmx3cXJUeUNmK1Uyby9RWmdl?= =?utf-8?B?Sm9yQzlRVUdGOU9XL1ZXRHczZDFqMGg3eTRRZndYYnJUUGlUVGlGQXMzNEtR?= =?utf-8?B?L2VzK29XOWVacWNGdndFa3BGaWxLdjVQN1g3NlBrNjlqeUhJT1hVeTlyaWtl?= =?utf-8?B?RG1aUTlmcElIRWRIRWFRb2ZVZmRqcGJjMHl1YUpjaGtOWktod0NTZmRKSGJR?= =?utf-8?B?TXJjL05GOFBBS0Q1eTBmV0UxQkM4aDR5aVVHRFlqbEVkUjRZQXVtQmp2b2dT?= =?utf-8?B?NDlzT3RyZTRrcGZkR0E2YTYxS3oybmNQTEtjWkFMZzFmM1B4TGZMRDA2cS9j?= =?utf-8?B?SjI2bG1DcG1mYmtlSzRSbWFGaE5xKzdEZEoxVkVEamxpVElmT3JOU04veDhq?= =?utf-8?B?aWFkaVd1WURFaFJraXBCdz09?= X-Microsoft-Exchange-Diagnostics: 1;BN6PR10MB1875;6:mnEK34MDyiYEnJ/ZVjUjCWSEqpQv898jLvVQ58JV0tfBWoJF6ZW9mxY95OdbKdjcfHb4NlFVhQRyaeZgcDdrRp7NxFNaT9T3mGk9ghyitxQQcJGsF9LIp5Rxsnrh0+JMVijyzO2xgHpwgLNL+eL+pYGM0fpdwt+sJviCIeO8sTw/AILhYU6b3n70wptPGIe7/PazZv6GmuZP3/jWNyIPzbAIRU9kD6ab+z8Irjm/v1pMCRDltH13XOvQTD60up3jUM+uUt4j/ahPQDCzMoBu8d32//9rSbLFVCwPIM+SoPi0DbVUc5R1KvjG+QoHh/u1CGj4n/NeBXik0oUiJdLxellT5cxXvBltLbHOQoR+ur0=;5:wkPUaPUG+WpQvt+cgdNUoTFacf7/talnU9JhLvV20Ji027oQW1eI8/5tPXMFocPx7cCxm4YMVQ3S3s2dWurUA0hAao69SqEmemd/BJYtxqCVSuoa1Epfoo8eGvjp3EqjH6NQ9ZjNav6hS8MnyHfID9NImjbMAe+Xk9fqT6q/vdQ=;24:rxACT49GrcEePMgLACdQV1V9GLwcuQ7omd9qujYEGQwKiFVOG3WPAFGWTOUbiLIew0jskpKsZyPaCpX03GGgPaD1TmfyMi52e5b4FIRCD+Y=;7:LpaPBQjNA4arYJznbccQnEPlw08wM0ugMo/E94oErG8anHU6j9zSy4t4c2mV7G66RBRucq9t66Wyl4Y/f1ss+oqqVb4mmy/3/7Js0amW75hkyyBc1nD5E9UJpaCtsHd7KOU7ZyjkDqACbKAbGnKKHsWbXU9tK/6IDBBVAQWqjbNB50PeSaKy/7FT7f9XfN3LER5CSmrPcwWzlupyiPajUz24FuSXTKSC5xa5FgpG4PjYAp37VU3mNswz4ShdN1vF SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: blackducksoftware.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Dec 2017 19:04:56.9610 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e3bc011c-c300-4b1b-6acc-08d53cdc3eef X-MS-Exchange-CrossTenant-Id: a8cff409-0d9d-48cf-b483-b7442cbeb39d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=a8cff409-0d9d-48cf-b483-b7442cbeb39d;Ip=[192.231.134.1];Helo=[ops-mail01.dc2.lan] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR10MB1875 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk On 12/06/17 12:19, Andres Freund wrote: > On 2017-12-06 12:14:24 -0500, Tom Lane wrote: >> Checking out hashint8() on random data shows no such obvious fault. >> You've apparently got a data set that exposes a weakness in hashint8, >> but it's not very clear what that is. > > It's intentionally designed to cause problems afaict: > > http://archives.postgresql.org/message-id/861b9f1f-cdc0-bc49-2595-80bc39c37dc3%40blackducksoftware.com Definitely not intentionally. My wording there could have been better; perhaps "we expect to get lots of duplicates in that column and we then need to find the unique values". Eventually, a substantial number of the duplicates will be deleted, but that's further along in the process. -- todd