MIME-Version: 1.0
References: <tencent_901D6A0152786410F0E00E72EC38432D0A09@qq.com>
 <28f47336-84e2-445e-8216-d1ce7d3ddc3e@iki.fi>
 <tencent_DCD98184361486D41AA430C8B0E0691D740A@qq.com>
 <tencent_E80D0FFA80C2AE21D4ABB0A7163EC54B0605@qq.com>
 <CAA5RZ0u=WDa-v+8Y_yE4WeRS8VtmcmQGuWVjZFCqZcXx-hEP3w@mail.gmail.com>
In-Reply-To: 
 <CAA5RZ0u=WDa-v+8Y_yE4WeRS8VtmcmQGuWVjZFCqZcXx-hEP3w@mail.gmail.com>
From: John Naylor <johncnaylorls@gmail.com>
Date: Thu, 4 Dec 2025 12:00:10 +0700
Message-ID: 
 <CANWCAZYKgc4xFbBXtePw_XY+a+Ao_-CxZ+Z8YWSrK2B1HqtdWg@mail.gmail.com>
Subject: Re: Support loser tree for k-way merge
To: Sami Imseih <samimseih@gmail.com>
Cc: cca5507 <cca5507@qq.com>, Heikki Linnakangas <hlinnaka@iki.fi>,
	pgsql-hackers <pgsql-hackers@lists.postgresql.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: 
 <https://www.postgresql.org/message-id/CANWCAZYKgc4xFbBXtePw_XY%2Ba%2BAo_-CxZ%2BZ8YWSrK2B1HqtdWg%40mail.gmail.com>
Precedence: bulk

On Thu, Dec 4, 2025 at 1:14=E2=80=AFAM Sami Imseih <samimseih@gmail.com> wr=
ote:
> Can we drive the decision for what to do based on optimizer
> stats, i.e. n_distinct and row counts? Not sure what the calculation woul=
d
> be specifically, but something else to consider.

It's happened multiple times before that someone proposes a change
that makes sorting faster on some inputs, but turns out to regress on
low cardinality (I've done it myself). It seems to be pretty hard not
to regress that case. Occasionally the author proposes to take
optimizer stats into account, and that was rejected because
cardinality stats are often wildly wrong.

Further, underestimation is far more common than overestimation, in
which case IIUC the planner would just continue to choose the existing
heap method.

> We can still provide the GUC to  override the optimizer decisions,
> but at least the optimizer, given up-to-date stats, may get it right most
> of the time.

I don't have much faith that people will properly set a GUC whose
effects depends on the input characteristics and memory settings.

The new method might be a better overall trade-off, but we'd need some
more comprehensive measurements to know what we're dealing with.

--
John Naylor
Amazon Web Services