Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.80) (envelope-from ) id 1WjpG9-0005yj-PW for pgsql-performance@arkaria.postgresql.org; Mon, 12 May 2014 12:23:10 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.80) (envelope-from ) id 1WjpG9-0005Xo-5L for pgsql-performance@arkaria.postgresql.org; Mon, 12 May 2014 12:23:09 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtp (Exim 4.80) (envelope-from ) id 1WjpG8-0005XF-8p for pgsql-performance@postgresql.org; Mon, 12 May 2014 12:23:08 +0000 Received: from mail-wi0-x232.google.com ([2a00:1450:400c:c05::232]) by magus.postgresql.org with esmtp (Exim 4.80) (envelope-from ) id 1WjpG4-0005MW-GZ for pgsql-performance@postgresql.org; Mon, 12 May 2014 12:23:07 +0000 Received: by mail-wi0-f178.google.com with SMTP id hm4so4358139wib.11 for ; Mon, 12 May 2014 05:23:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=user-agent:date:subject:from:to:cc:message-id:thread-topic :references:in-reply-to:mime-version:content-type; bh=6YczS+1PiG0ZhZpLzKqAIl0MfFkeC2Aq0a6S39kMEgY=; b=NQKhcS8Jw8pGVtibvozEKeIdc6DzW1I65XvIAhe6rVeezJ3WpZLLqimZKqg6jzv/Kg YJ1PQsn7Vs6XrLVZ6x322qM9VwxsqiClD7xkrPcCcgRQIutlLKqCvrBArD+WvuUA+PVK OJQW5gTNIMh9RTRd39zBe2kIqmlWpXE8o/62n/YXlb5ZDgNEw13xvPSmdV0b0cqr8cru qon3S0GKXUUZSuk7PanwhgTaa9waTrszhO9mU8tzPgqkBaZU8J/NuCrZG2sdc06thE8C 2JHkh46mwSmhbedAq5Rm3Q+tk5neQWx5lwaPFqxDVEd7Wg72bJ512Pm0iptAnGW2J/Cm GRaw== X-Received: by 10.194.89.168 with SMTP id bp8mr1334242wjb.73.1399897383338; Mon, 12 May 2014 05:23:03 -0700 (PDT) Received: from [172.16.112.47] (london.blinkboxmusic.com. [31.221.105.130]) by mx.google.com with ESMTPSA id gx5sm15819941wib.23.2014.05.12.05.22.58 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Mon, 12 May 2014 05:23:01 -0700 (PDT) User-Agent: Microsoft-MacOutlook/14.4.1.140326 Date: Mon, 12 May 2014 13:22:51 +0100 Subject: Re: 9.2.4 vs 9.3.0 query planning (sort merge join vs hash join) From: Tim Kane To: Tom Lane CC: "pgsql-performance@postgresql.org" Message-ID: Thread-Topic: [PERFORM] 9.2.4 vs 9.3.0 query planning (sort merge join vs hash join) References: <17739.1399893780@sss.pgh.pa.us> In-Reply-To: <17739.1399893780@sss.pgh.pa.us> Mime-version: 1.0 Content-type: multipart/alternative; boundary="B_3482745780_64553334" X-Pg-Spam-Score: -2.0 (--) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgsql-performance Precedence: bulk Sender: pgsql-performance-owner@postgresql.org > This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. --B_3482745780_64553334 Content-type: text/plain; charset="UTF-8" Content-transfer-encoding: quoted-printable Hmm. Interesting. Thanks Tom, it does indeed look like the planner is evaluating and excludin= g the hashed-join plan as having a higher cost. I can see this by setting enable_mergejoin=3D0. I think this may play against other aspects of the query (though only marginally), so I can=E2=80=99t really compare the resulting cost metrics =E2=80=93 but they=E2=80=99re certainly close. I=E2=80=99ve just now played a little more with work_mem. I had already tried increasing work_mem to all kinds of obscene levels. Oddly, it seems the solution here is in fact to *reduce* work_mem in order to elicit the preferred hash-join based plan. In fact, I needed to reduce it to as low as 64MB for this behaviour =E2=80=93 whi= ch seems counter-intuitive. These are not small queries, so I previously had i= t pushed up to 6GB for these tasks. For smaller datasets,I need to reduce work_mem further still in order to obtain a hash-join plan =E2=80=93 though the difference in execution time becomes less of a problem at this size. Tim From: Tom Lane Date: Monday, 12 May 2014 12:23 To: Tim Kane Cc: "pgsql-performance@postgresql.org" Subject: Re: [PERFORM] 9.2.4 vs 9.3.0 query planning (sort merge join vs hash join) Tim Kane writes: > Is there a propensity for 9.2.4 to prefer a sort-merge-join, in place of= a > hash join? Not particularly; I don't think there's any actual difference in the cost estimation equations between 9.2 and 9.3. The two plans you show are close enough in estimated cost that the ordering of their costs might be coming out differently just as a matter of random variation in statistics. It'd be worth double-checking the work_mem setting on both systems, though, as (IIRC) an undersized work_mem hurts the estimate for hashes more than for sorts. regards, tom lane --B_3482745780_64553334 Content-type: text/html; charset="UTF-8" Content-transfer-encoding: quoted-printable

Hmm. Interesti= ng.

Thanks Tom, it does indeed look like the p= lanner is evaluating and excluding the hashed-join plan as having a higher c= ost. I can see this by setting enable_mergejoin=3D0.
I think this ma= y play against other aspects of the query (though only marginally), so I can= ’t really compare the resulting cost metrics – but they’re= certainly close.


I’ve = just now played a little more with work_mem.  I had already tried incre= asing work_mem to all kinds of obscene levels.

Oddl= y, it seems the solution here is in fact to *reduce* work_mem in order to el= icit the preferred hash-join based plan.
In fact, I needed to redu= ce it to as low as 64MB for this behaviour – which seems counter-intui= tive. These are not small queries, so I previously had it pushed up to 6GB f= or these tasks.

For smaller datasets,I need t= o reduce work_mem further still in order to obtain a hash-join plan – = though the difference in execution time becomes less of a problem at this si= ze.

Tim


From: Tom Lane <tgl@sss.pgh.pa.us>
Date: Monday, 12 May 2014 12:23
To: = Tim Kane <tim.kane@gmail.com><= br>Cc: "pgsql-performance@postgresql.org" <pgsql-performance@postgresql.org>= ;
Subject: Re: [PERFORM] 9.2.4 vs = 9.3.0 query planning (sort merge join vs hash join)

=
Tim Kane <tim.kane@gma= il.com> writes:
<= div> Is there a propensity for 9.2.4 to prefer a sort-merge-join, in place o= f a
hash join?

Not particul= arly; I don't think there's any actual difference in the cost
esti= mation equations between 9.2 and 9.3.  The two plans you show are<= /div>
close enough in estimated cost that the ordering of their costs mi= ght be
coming out differently just as a matter of random variation= in statistics.

It'd be worth double-checking the w= ork_mem setting on both systems,
though, as (IIRC) an undersized w= ork_mem hurts the estimate for hashes
more than for sorts.

regards, tom lane

--B_3482745780_64553334--