Subject: Re: Change default of jit to off

From: Christoph Berg <[email-masked]>
Date: Tue, 3 Feb 2026 11:29:13 +0100

Re: Euler Taveira
> > +1 on disabling jit by default. At the FOSDEM Postgres developer meeting
> > consensus was hugely in favor of changing the default. So attached is a
> > trivial patch that does this.

I'm also in favor of disabling it by default. It's mostly hurting
people (or use cases) in OLTP workloads where it's just surprising
that JIT kicks in, while in OLAP workloads, people would usually know
they have a use for JIT and could enable it.

There are several layers of "default" here:
1) the default when compiling without extra options
2) the default when compiling with LLVM
3) the default in the deb/rpm/... packages with just "postgresql" installed
4) the default in the deb/rpm/... packages with the JIT package installed
5) the default that cloud providers set when using these/their own packages

The setting for 1) is off, so you might argue that for all other
cases, someone has made a conscious choice to enable it for the other
options, but for 3/5) and likely also for 4), that decision wasn't
made by the user actually using the database, so they don't have any
influence on it. In these cases, disabling it would remove the
"surprise" moment.

That leaves 2), but I would argue that all the cases should behave the
same. If different installs have different defaults, that's hard to
communicate.

> You can argue that the majority of the workloads are
> not composed by analytic queries. Fine. That's not an excuse to perform poorly
> with a small percentage of these analytic queries.

It's neither an excuse to perform poorly on the OLTP queries that got
JITed when they should not have.

> Since there are some reports exposing these issues and articles discussing it,
> I would suggest improving the documentation stating that there are issues in
> this area and that they will be resolved in the future. (The documentation
> already stated the common issue -- short queries -- in the first paragraph [2].
> Maybe we need to be more incisive.)

Documentation won't help here. This is always biting users by
surprise. Telling them that their non-analytic workload performs
poorly because they didn't read the JIT chapter when it shouldn't be
relevant to them is quite weird.

> Instead, of using the argument that JIT hurts short queries, I would like to
> see other alternative solutions for this JIT issue. The v15 added some columns
> to pg_stat_statements that helps us understand if JIT is hurting your queries.
> I don't have numbers but instead of turning JIT off we should adopt a less
> invasive solution like increasing the thresholds that triggers it --
> jit_*_above_cost (probably using some pg_stat_statements reports). It is not a
> definitive fix, JIT needs improvement and an idea like making JIT more granular
> [3] could restore confidence in it.

I've been pondering this as well, but I think we aren't there yet. It
will still make the compiler jump in in situations where it shouldn't
(maybe the cost estimates are just wildly off, but often the planner
still finds a good OLTP plan). We want predictability here.

> Although I don't have plans to hack on JIT code but
> reports may encourage a few developers to propose changes in this area.

We'd need people to actually work out the cost parameters. :)

Christoph

---

From: Adrien Nayrat <[email-masked]>
Date: Tue, 3 Feb 2026 16:46:38 +0100

Hello

> 5) the default that cloud providers set when using these/their own packages

Just to mention, it is disabled on most cloud provider :

* AWS RDS : off (but available)
* Google Cloud SQL : off (not supported) 
https://docs.cloud.google.com/sql/docs/postgres/features#unsupported-features-for-postgres
* Microsoft Aure : off 
https://learn.microsoft.com/fr-fr/azure/postgresql/server-parameters/param-query-tuning-planner-options?pivots=postgresql-18#jit
* Aiven : off https://aiven.io/docs/products/postgresql/howto/enable-jit

---

From: Greg Sabino Mullane <[email-masked]>
Date: Tue, 3 Feb 2026 12:03:51 -0500

Given that so many places are already disabling it, +1 to disabling by
default - unless someone can come up with a costing tweak so it doesn't
fire when it shouldn't - but right now that seems something only humans can
truly determine.

---

From: Andres Freund <[email-masked]>
Date: Tue, 3 Feb 2026 12:13:46 -0500

Hi,

+1 for disabling it by default. Particularly with partitioning having become
much more common since jit compilation was added, combined with LLVM getting a
lot slower over time, that's unfortunately the right call until some
substantial improvements are made.

> fire when it shouldn't - but right now that seems something only humans can

I think it needs more than a costing tweak. The most important thing would be
to get caching (there's progress, albeit very slow one, towards that), so the
downside of JIT compilation doesn't hit you over and over.

Relatedly, we often end up with the almost-same expression being jit compiled
many times in partitioned workloads, which is one of the main sources of high
jit compilation times.

We also need to just increase the benefit of JIT compilation further, the code
we generate leaves a *lot* on the table, particularly for more complicated
expressions, where the gain also can be the biggest.

Greetings,

Andres Freund

---

From: Jelte Fennema-Nio <[email-masked]>
Date: Fri, 13 Feb 2026 15:57:18 +0100

> substantial improvements are made.

I'm marking this as ready for committer.

---

From: Pierre Ducroquet <[email-masked]>
Date: Mon, 23 Mar 2026 06:33:33 +0000

Le vendredi 20 mars 2026 à 5:25 PM, Tom Lane <[email-masked]> a écrit :

> Tomas Vondra <[email-masked]> writes:
> > ISTM there's a clear consensus to get this committed for PG19, so
> > barring objections I'll take care of that in the next couple days.
> > Unless someone else wants to ...
> 
> +1
> 
> > Another option would be to leave that for mid-beta, which is where we
> > tweaked the io_method GUCs last year. But we did that to get some
> > testing for 'worker' (in case we revert to 'sync'), and we don't need
> > that for jit.
> 
> Doesn't seem like something to change mid-beta.  If it makes anyone
> unhappy, we'd best find out sooner not later.

I've not seen any feedback on my "counter"-proposal: switch jit_tuple_deforming to off by default. Sure, for the perfect llvmjit use cases this will reduce the performance benefits, but it will remove most if not all the problematic queries (for instance queries running on many partitions, adding/moving columns leading to explosions in compilation time...)
Of course if there are other troublesome situations, I would love being proven wrong.

---

From: "Jelte Fennema-Nio" <[email-masked]>
Date: Tue, 24 Mar 2026 15:32:36 +0100

On Mon Mar 23, 2026 at 9:50 AM CET, Jelte Fennema-Nio wrote:
> I think Andres' reply also applied to that suggestion. Partitioning
> really messes up jit for expressions too. I was able to pretty easily
> create some bad cases even with jit_tuple_deforming set to off. A
> query involving many partitions, but with relatively few rows per
> partition, jit became ~54x slower when using -O2 (743ms vs 40318ms).
> And even with -O0, jit is still ~3x slower (221ms vs 711ms). Full
> script and results attached.

Resending the above with a compressed attachment, because it was too
large for the mailing list.

---

From: Andres Freund <[email-masked]>
Date: Tue, 24 Mar 2026 17:56:39 -0400

Hi,

On 2026-03-23 06:33:33 +0000, Pierre Ducroquet wrote:
> I've not seen any feedback on my "counter"-proposal: switch
> jit_tuple_deforming to off by default. Sure, for the perfect llvmjit use
> cases this will reduce the performance benefits, but it will remove most if
> not all the problematic queries (for instance queries running on many
> partitions, adding/moving columns leading to explosions in compilation
> time...)  Of course if there are other troublesome situations, I would love
> being proven wrong.

I doubt that that addresses the problem in any meaningful way. In nearly all
the cases I've looked at expression compilation completely dominates the cost,
due to being instantiated for every partition etc. So I'm rather surprised to
see this claim?

We should add the function deduplication pass for O0, to reduce the cost of
tuple deforming when accessing many partitions, but in my measurements that
isn't the critical path right now.

Greetings,

Andres Freund

---

From: Tomas Vondra <[email-masked]>
Date: Wed, 25 Mar 2026 23:10:25 +0100

> due to being instantiated for every partition etc. So I'm rather surprised to

That's my understanding too. I did search in the archives for
performance issues related to JIT, and those are the first two I found:

1)
https://www.postgresql.org/message-id/[email-masked]

2)
https://www.postgresql.org/message-id/flat/LEXPR01MB09897821BA2CDDB70225EF7BE79C0%40LEXPR01MB0989.DEUPRD01.PROD.OUTLOOK.DE

Both have all the JIT stuff enabled, and the timings are dominated by
Optimization:

* Timing: Generation 15.707 ms, Inlining 4.688 ms, Optimization
652.021 ms, Emission 939.556 ms, Total 1611.973 ms

* Timing: Generation 27.833 ms, Inlining 117.093 ms, Optimization
3917.001 ms, Emission 2027.251 ms, Total 6089.177 ms

Or am I mistaken and it could still be due to deforming? It'd be a bit
silly to disable jit_tuple_deforming=off, only to find it's not enough
and have to disable more stuff for PG20.

Furthermore, jit_tuple_deforming is marked as DEVELOPER_OPTIONS, and is
not in the sample. Wouldn't that be a bit confusing/surprising, if we
disable just that, as it's not in the sample?

regards

-- 
Tomas Vondra

---

From: Tomas Vondra <[email-masked]>
Date: Thu, 2 Apr 2026 13:51:46 +0200

Pushed, after rewording the commit message a little bit, and removing
the sentence in the docs saying

    (before PostgreSQL 19 the default was on)

We've changed the defaults for a number of GUCs over the years (and some
of the GUCs were more critical than "jit"), and we have not added
similar comments to those. I haven't found a single GUC with such
comment, actually. So why have it for "jit"?

regards

-- 
Tomas Vondra

---

From: Thomas Munro <[email-masked]>
Date: Fri, 3 Apr 2026 01:50:30 +1300

>     (before PostgreSQL 19 the default was on)

How can we keep it enabled in the build farm?

---

From: Andrew Dunstan <[email-masked]>
Date: Thu, 2 Apr 2026 09:10:57 -0400

> How can we keep it enabled in the build farm?

No easy way to do it globally - I guess we could default it to on in the 
next client release. Individual owners can enable it in their buildfarm 
configs, in the extra_config section.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

---

From: Greg Sabino Mullane <[email-masked]>
Date: Thu, 2 Apr 2026 09:46:56 -0400

Probably want a new buildfarm flag/icon for it then - maybe a monkey
wrench? :)

---

From: Andrew Dunstan <[email-masked]>
Date: Thu, 2 Apr 2026 10:01:54 -0400

> Probably want a new buildfarm flag/icon for it then - maybe a monkey 

This things are for configure options, not runtime options, although we 
could expand that, I guess. It's a SMOC.

cheers

andrew

---

From: Tomas Vondra <[email-masked]>
Date: Thu, 2 Apr 2026 21:46:40 +0200

> configs, in the extra_config section.

That's a good idea. Maybe we should ping the buildfarm owners (those
that are already running --with-llvm) to add jit=on.

I'll do that on my two clang animals dikkop/jay, and maybe on some of
the gcc ones too. But I'll wait a bit, I don't want to upset them right
before feature freeze.

regards

-- 
Tomas Vondra

---

From: Tomas Vondra <[email-masked]>
Date: Mon, 13 Apr 2026 00:27:33 +0200

> the gcc ones too. But I'll wait a bit, I don't want to upset them right

FWIW I've now done this on my 5 animals - dikkop, turaco, widowbird, jay
and hippopotamus. None of them was even building with LLVM before. On
the last two it was a bit of a struggle, due to having old gcc and LLVM.

regards

-- 
Tomas Vondra