From: Justin Pryzby <pryzby@telsasoft.com>
To: Andres Freund <andres@anarazel.de>
Cc: pgsql-hackers@postgresql.org, Tom Lane <tgl@sss.pgh.pa.us>
Subject: Re: terminate called after throwing an instance of 'std::bad_alloc'
Date: Mon, 19 Apr 2021 12:41:36 -0500
Message-ID: <20210419174136.GJ7256@telsasoft.com> (raw)
In-Reply-To: <20210419164130.byegpfrw46mzagcu@alap3.anarazel.de>
References: <20201003230149.mtd7fjsjwgii3jv7@alap3.anarazel.de>
<20201003233045.GM8476@telsasoft.com>
<20201004020127.zbrol2ypu6tg2bei@alap3.anarazel.de>
<20201007230242.rnwsbnxywvthef3w@alap3.anarazel.de>
<20201218235607.GC30237@telsasoft.com>
<20210417021755.xdcqaqqt4y46wzt7@alap3.anarazel.de>
<20210417024854.GL3315@telsasoft.com>
<20210417031837.GM3315@telsasoft.com>
<20210418001324.GP3315@telsasoft.com>
<20210419164130.byegpfrw46mzagcu@alap3.anarazel.de>
On Mon, Apr 19, 2021 at 09:41:30AM -0700, Andres Freund wrote:
> On 2021-04-17 19:13:24 -0500, Justin Pryzby wrote:
> > I'm now realizing that that's RAM use for a single query, not from continuous
> > leaks across multiple queries.
>
> What's the memory usage with inlining disabled, and whats the usage
> without jit?
PGOPTIONS="-c jit=off -c jit_inline_above_cost=0 -c jit_above_cost=0 -c client_min_messages=debug -c log_executor_stats=on" python3 `pwd`/jitleak.py
! [5.272604 s user, 0.329912 s system total]
! 120948 kB max resident size
PGOPTIONS="-c jit=on -c jit_inline_above_cost=0 -c jit_above_cost=0 -c client_min_messages=debug -c log_executor_stats=on" python3 `pwd`/jitleak.py
DEBUG: time to inline: 0.003s, opt: 0.000s, emit: 0.003s
... repeated many times ...
! [585.090087 s user, 31.835228 s system total]
! 4639556 kB max resident size
PGOPTIONS="-c jit=on -c jit_inline_above_cost=-1 -c jit_above_cost=0 -c client_min_messages=debug -c log_executor_stats=on" python3 `pwd`/jitleak.py
DEBUG: time to inline: 0.000s, opt: 0.000s, emit: 0.003s
... repeated many times ... which is confusing, since inlining is disabled ...
! [318.556489 s user, 8.413007 s system total]
! 159436 kB max resident size
> > Maybe we're missing this condition somehow ?
> > if (llvm_jit_context_in_use_count == 0 &&
>
> Do you utilize cursors or such?
Not in this backend/process.
I believe postgis/mapserver uses cursors in a separate process, though...
Our report runs like:
SELECT large_object_oid FROM ... WHERE name=$1 AND user=$2 ... FOR UPDATE -- our reports are stored as LOs and read by readlo
begin;
SET enable_nestloop=off; -- I'm sorry to say but nothing else has avoids occasional pathological query plans
SELECT report text...;
rollback;
> > Also, I just hit this assertion by cancelling the query with ^C / sigint. But
> > I don't have a reprodcer for it.
> >
> > < 2021-04-17 19:14:23.509 ADT telsasoft >PANIC: LLVMJitContext in use count not 0 at exit (is 1)
>
> Hm, I'll try to see how that happens - if not, do you have a backtrace?
This may be because I backpatched to v13, which had one conflict (0a2bc5d61).
Maybe this is relevant, too: bab150045
If you could provide a patch for v13, it's a good bet there's no issue that I
didn't cause.
I'm able to reproduce the crash like this, with my patch to v13, but not your
patch on master.
python3 -c "import pg; db=pg.DB('dbname=postgres'); db.query('SET jit_above_cost=0; SET jit_inline_above_cost=0; SET jit=on; SET client_min_messages=debug'); db.query('begin'); db.query_formatted('SELECT 1 FROM generate_series(1,99)a WHERE a=%s', [1], inline=False);"
--
Justin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: pryzby@telsasoft.com, andres@anarazel.de, tgl@sss.pgh.pa.us
Subject: Re: terminate called after throwing an instance of 'std::bad_alloc'
In-Reply-To: <20210419174136.GJ7256@telsasoft.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox