From: Peter Geoghegan <pg@heroku.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Jeff Janes <jeff.janes@gmail.com>
Cc: Pg Hackers <pgsql-hackers@postgresql.org>
Cc: Heikki Linnakangas <hlinnaka@iki.fi>
Cc: Simon Riggs <simon@2ndquadrant.com>
Subject: Re: Using quicksort for every external sort run
Date: Thu, 19 Nov 2015 14:53:47 -0800
Message-ID: <CAM3SWZTR3UE2FEaaAiDbs7N31Td1KufQDAXKeVi4hDMva0PRjQ@mail.gmail.com> (raw)
In-Reply-To: <CA+Tgmob9ssG0XB4-GzPARCe0C8gXzHBCrAzNyout1oWj+9Nc9Q@mail.gmail.com>
References: <CAM3SWZQVSpNeHHKKq-rjJddOcbpdmyHDJUMBBL2-AP2R+4YCHg@mail.gmail.com>
<CAMkU=1ym2N-v+3=aMm9m-h1A941gQV7CgM9HJuwhRFu-8tHFaw@mail.gmail.com>
<CAM3SWZQS==-K+SpLsmPdzQ=nrXKMhyNDMELfJKWbpKx0a1+sdg@mail.gmail.com>
<CA+TgmoakcV5FzZwo+WNwbpnXwbY3N+qVs_zKdmE_OYmfivWnkQ@mail.gmail.com>
<CAM3SWZTX5=nHxPpogPirQsH4cR+BpQS6r7Ktax0HMQiNLf-1qA@mail.gmail.com>
<CA+Tgmob9ssG0XB4-GzPARCe0C8gXzHBCrAzNyout1oWj+9Nc9Q@mail.gmail.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
On Thu, Nov 19, 2015 at 2:35 PM, Robert Haas <robertmhaas@gmail.com> wrote:
> OK, so reversing this analysis, with the default work_mem of 4MB, we'd
> need a multi-pass merge for more than 235MB/4 = 58MB of data. That is
> very, very far from being a can't-happen scenario, and I would not at
> all think it would be acceptable to ignore such a case.
> I think you need to revisit your assumptions here.
Which assumption? Are we talking about multipass_warning, or my patch
series in general? Obviously those are two very differently things. As
I've said, we could address the visibility aspect of this differently.
I'm fine with that.
I'll now talk about my patch series in general -- the actual
consequences of not avoiding a single pass merge phase when the master
branch would have done so.
The latter 16MB work_mem example query/table is still faster with a
12MB work_mem than master, even with multiple passes. Quite a bit
faster, in fact: about 37 seconds on master, to about 24.7 seconds
with the patches (same for higher settings short of 16MB).
Now, that's probably slightly unfair on the master branch, because the
patches still have the benefit of the memory pooling during the merge
phase, which is nothing to do with what we're talking about, and
because my laptop still has plenty of ram.
I should point out that there is no evidence that any case has been
regressed, let alone written off entirely or ignored. I looked. I
probably have not been completely exhaustive, and I'd be willing to
believe there is something that I've missed, but it's still quite
possible that there is no downside to any of this.
--
Peter Geoghegan
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: pg@heroku.com, robertmhaas@gmail.com, jeff.janes@gmail.com, hlinnaka@iki.fi, simon@2ndquadrant.com
Subject: Re: Using quicksort for every external sort run
In-Reply-To: <CAM3SWZTR3UE2FEaaAiDbs7N31Td1KufQDAXKeVi4hDMva0PRjQ@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox