Received: from maia.hub.org (maia-5.hub.org [200.46.204.29]) by mail.postgresql.org (Postfix) with ESMTP id B84B91337B7A for ; Thu, 13 Jan 2011 18:47:38 -0400 (AST) Received: from mail.postgresql.org ([200.46.204.86]) by maia.hub.org (mx1.hub.org [200.46.204.29]) (amavisd-maia, port 10024) with ESMTP id 69703-07 for ; Thu, 13 Jan 2011 22:47:32 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from marajade.localdomain (vcs5.camavision.com [63.228.164.252]) by mail.postgresql.org (Postfix) with ESMTP id 3BFE91337B5E for ; Thu, 13 Jan 2011 18:47:29 -0400 (AST) Received: from [192.168.10.84] (unknown [192.168.10.84]) (Authenticated sender: andy) by marajade.localdomain (Postfix) with ESMTPA id 3307B4294E; Thu, 13 Jan 2011 16:47:29 -0600 (CST) Message-ID: <4D2F8103.7010805@squeakycode.net> Date: Thu, 13 Jan 2011 16:47:31 -0600 From: Andy Colson User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.13) Gecko/20101207 Lightning/1.0b2 Thunderbird/3.1.7 MIME-Version: 1.0 To: Robert Haas CC: Tom Lane , Jon Nelson , pgsql-performance@postgresql.org Subject: Re: queries with lots of UNIONed relations References: <19302.1294938802@sss.pgh.pa.us> <4879.1294949144@sss.pgh.pa.us> <11563.1294957582@sss.pgh.pa.us> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=-1.9 tagged_above=-10 required=5 tests=BAYES_00=-1.9 X-Spam-Level: X-Archive-Number: 201101/87 X-Sequence-Number: 41874 On 1/13/2011 4:42 PM, Robert Haas wrote: > On Thu, Jan 13, 2011 at 5:41 PM, Robert Haas wrote: >> On Thu, Jan 13, 2011 at 5:26 PM, Tom Lane wrote: >>> Robert Haas writes: >>>> On Thu, Jan 13, 2011 at 3:12 PM, Jon Nelson wrote: >>>>> I still think that having UNION do de-duplication of each contributory >>>>> relation is a beneficial thing to consider -- especially if postgresql >>>>> thinks the uniqueness is not very high. >>> >>>> This might be worth a TODO. >>> >>> I don't believe there is any case where hashing each individual relation >>> is a win compared to hashing them all together. If the optimizer were >>> smart enough to be considering the situation as a whole, it would always >>> do the latter. >> >> You might be right, but I'm not sure. Suppose that there are 100 >> inheritance children, and each has 10,000 distinct values, but none of >> them are common between the tables. In that situation, de-duplicating >> each individual table requires a hash table that can hold 10,000 >> entries. But deduplicating everything at once requires a hash table >> that can hold 1,000,000 entries. >> >> Or am I all wet? > > Yeah, I'm all wet, because you'd still have to re-de-duplicate at the > end. But then why did the OP get a speedup? *scratches head* > Because it all fix it memory and didnt swap to disk? -Andy