pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedspurious function execution in prepared statements.
7+ messages / 4 participants
[nested] [flat]
* spurious function execution in prepared statements.
@ 2004-09-30 13:45 Merlin Moncure <merlin.moncure@rcsonline.com>
0 siblings, 3 replies; 7+ messages in thread
From: Merlin Moncure @ 2004-09-30 13:45 UTC (permalink / raw)
To: pgsql-performance; +Cc: pgsql-hackers@postgresql.org
OK, I have a situation that might be a performance problem, a bug, or an
unavoidable consequence of using prepared statements. The short version
is that I am getting function executions for rows not returned in a
result set when they are in a prepared statement.
In other words, I have a query:
select f(t.c) from t where [boolean expr on t] limit 1;
because of the limit phrase, obviously, at most one record is returned
and f executes at most once regardless of the plan used (in practice,
sometimes index, sometimes seq_scan.
Now, if the same query is executed as a prepared statement,
prepare ps(...) as select f(t.c) from t where [expr] limit 1;
execute ps;
now, if ps ends up using a index scan on t, everything is ok. However,
if ps does a seqscan, f executes for every row on t examined until the
[expr] criteria is met. Is this a bug? If necessary I should be able
to set up a reproducible example. The easy workaround is to not use
prepared statements in these situations, but I need to be able to
guarantee that f only executes once (even if that means exploring
subqueries).
Merlin
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: [HACKERS] spurious function execution in prepared statements.
@ 2004-09-30 14:13 Michael Adler <adler@pobox.com>
parent: Merlin Moncure <merlin.moncure@rcsonline.com>
2 siblings, 0 replies; 7+ messages in thread
From: Michael Adler @ 2004-09-30 14:13 UTC (permalink / raw)
To: Merlin Moncure <merlin.moncure@rcsonline.com>; +Cc: pgsql-performance; pgsql-hackers@postgresql.org
On Thu, Sep 30, 2004 at 09:45:51AM -0400, Merlin Moncure wrote:
> Now, if the same query is executed as a prepared statement,
> prepare ps(...) as select f(t.c) from t where [expr] limit 1;
> execute ps;
>
> now, if ps ends up using a index scan on t, everything is ok. However,
> if ps does a seqscan, f executes for every row on t examined until the
> [expr] criteria is met. Is this a bug? If necessary I should be able
> to set up a reproducible example. The easy workaround is to not use
> prepared statements in these situations, but I need to be able to
> guarantee that f only executes once (even if that means exploring
> subqueries).
Here's another workaround that may let you use a prepared statement:
prepare ps(...) as
select f(c) from (select c from t where [expr] limit 1) as t1
-Mike
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: [HACKERS] spurious function execution in prepared statements.
@ 2004-09-30 14:19 Merlin Moncure <merlin.moncure@rcsonline.com>
0 siblings, 0 replies; 7+ messages in thread
From: Merlin Moncure @ 2004-09-30 14:19 UTC (permalink / raw)
To: Michael Adler <adler@pobox.com>; +Cc: pgsql-performance; pgsql-hackers@postgresql.org
> Here's another workaround that may let you use a prepared statement:
>
> prepare ps(...) as
> select f(c) from (select c from t where [expr] limit 1) as t1
>
> -Mike
I was just exploring that. In fact, the problem is not limited to
prepared statements...it's just that they are more likely to run a
seqscan so I noticed it there first. Since I am in a situation where I
need very strict control over when and why f gets executed, I pretty
much have to go with the subquery option.
That said, it just seems that out of result set excecutions of f should
be in violation of something...
Merlin
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: spurious function execution in prepared statements.
@ 2004-09-30 14:45 Stephan Szabo <sszabo@megazone.bigpanda.com>
parent: Merlin Moncure <merlin.moncure@rcsonline.com>
2 siblings, 0 replies; 7+ messages in thread
From: Stephan Szabo @ 2004-09-30 14:45 UTC (permalink / raw)
To: Merlin Moncure <merlin.moncure@rcsonline.com>; +Cc: pgsql-performance; pgsql-hackers@postgresql.org
On Thu, 30 Sep 2004, Merlin Moncure wrote:
> OK, I have a situation that might be a performance problem, a bug, or an
> unavoidable consequence of using prepared statements. The short version
> is that I am getting function executions for rows not returned in a
> result set when they are in a prepared statement.
>
> In other words, I have a query:
> select f(t.c) from t where [boolean expr on t] limit 1;
An actual boolean expr on t? Or on a column in t?
> because of the limit phrase, obviously, at most one record is returned
> and f executes at most once regardless of the plan used (in practice,
> sometimes index, sometimes seq_scan.
>
> Now, if the same query is executed as a prepared statement,
> prepare ps(...) as select f(t.c) from t where [expr] limit 1;
> execute ps;
>
> now, if ps ends up using a index scan on t, everything is ok. However,
> if ps does a seqscan, f executes for every row on t examined until the
> [expr] criteria is met. Is this a bug? If necessary I should be able
> to set up a reproducible example. The easy workaround is to not use
> prepared statements in these situations, but I need to be able to
> guarantee that f only executes once (even if that means exploring
> subqueries).
I think a reproducible example would be good. Simple attempts to duplicate
this on 8.0b2 have failed for me, unless I'm using order by.
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: [HACKERS] spurious function execution in prepared statements.
@ 2004-09-30 14:54 Tom Lane <tgl@sss.pgh.pa.us>
parent: Merlin Moncure <merlin.moncure@rcsonline.com>
2 siblings, 0 replies; 7+ messages in thread
From: Tom Lane @ 2004-09-30 14:54 UTC (permalink / raw)
To: Merlin Moncure <merlin.moncure@rcsonline.com>; +Cc: pgsql-performance; pgsql-hackers@postgresql.org
"Merlin Moncure" <merlin.moncure@rcsonline.com> writes:
> now, if ps ends up using a index scan on t, everything is ok. However,
> if ps does a seqscan, f executes for every row on t examined until the
> [expr] criteria is met. Is this a bug?
Works for me.
regression=# create function f(int) returns int as '
regression'# begin
regression'# raise notice ''f(%)'', $1;
regression'# return $1;
regression'# end' language plpgsql;
CREATE FUNCTION
regression=# select f(unique2) from tenk1 where unique2%2 = 1 limit 2;
NOTICE: f(1)
NOTICE: f(3)
f
---
1
3
(2 rows)
regression=# prepare ps as
regression-# select f(unique2) from tenk1 where unique2%2 = 1 limit 2;
PREPARE
regression=# execute ps;
NOTICE: f(1)
NOTICE: f(3)
f
---
1
3
(2 rows)
regression=#
You sure you aren't using f() in the WHERE clause?
regards, tom lane
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: spurious function execution in prepared statements.
@ 2004-09-30 14:58 Merlin Moncure <merlin.moncure@rcsonline.com>
0 siblings, 1 reply; 7+ messages in thread
From: Merlin Moncure @ 2004-09-30 14:58 UTC (permalink / raw)
To: Stephan Szabo <sszabo@megazone.bigpanda.com>; +Cc: pgsql-performance; pgsql-hackers@postgresql.org; Tom Lane <tgl@sss.pgh.pa.us>
Stephan Szabo wrote:
> On Thu, 30 Sep 2004, Merlin Moncure wrote:
>
> > OK, I have a situation that might be a performance problem, a bug,
or an
> > unavoidable consequence of using prepared statements. The short
version
> > is that I am getting function executions for rows not returned in a
> > result set when they are in a prepared statement.
> An actual boolean expr on t? Or on a column in t?
[...]
> I think a reproducible example would be good. Simple attempts to
duplicate
> this on 8.0b2 have failed for me, unless I'm using order by.
Note: I confirmed that breaking out the 'where' part of the query into
subquery suppresses the behavior.
Here is the actual query:
select lock_cuid(id), *
from data3.wclaim_line_file
where wcl_vin_no >= '32-MHAB-C-X-7243' and
(wcl_vin_no > '32-MHAB-C-X-7243' or wcl_claim_no >=
001) and
(wcl_vin_no > '32-MHAB-C-X-7243' or wcl_claim_no >
001 or id > 2671212)
order by wcl_vin_no, wcl_claim_no, id
limit 1
Here is the prepared statement declaration:
prepare data3_read_next_wclaim_line_file_1_lock (character varying,
numeric, int8, numeric)
as select lock_cuid(id), *
from data3.wclaim_line_file
where wcl_vin_no >= $1 and
(wcl_vin_no > $1 or wcl_claim_no >= $2) and
(wcl_vin_no > $1 or wcl_claim_no > $2 or id > $3)
order by wcl_vin_no, wcl_claim_no, id limit $4
Here is the plan when it runs lock_cuid repeatedly (aside: disabling
seqscans causes an index plan, but that's not the point):
esp=# explain execute data3_read_next_wclaim_line_file_1_lock
('32-MHAB-C-X-7243', 001, 2671212, 1);
QUERY PLAN
------------------------------------------------------------------------
----------------------------
------------------------------------------------------------------------
----------------------------
--------------------------------
Limit (cost=13108.95..13162.93 rows=21592 width=260)
-> Sort (cost=13108.95..13162.93 rows=21592 width=260)
Sort Key: wcl_vin_no, wcl_claim_no, id
-> Seq Scan on wclaim_line_file (cost=0.00..11554.52
rows=21592 width=260)
Filter: (((wcl_vin_no)::text >= ($1)::text) AND
(((wcl_vin_no)::text > ($1)::text) OR
((wcl_claim_no)::numeric >= $2)) AND (((wcl_vin_no)::text > ($1)::text)
OR ((wcl_claim_no)::numeric
> $2) OR ((id)::bigint > $3)))
(5 rows)
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: spurious function execution in prepared statements.
@ 2004-09-30 15:02 Tom Lane <tgl@sss.pgh.pa.us>
parent: Merlin Moncure <merlin.moncure@rcsonline.com>
0 siblings, 0 replies; 7+ messages in thread
From: Tom Lane @ 2004-09-30 15:02 UTC (permalink / raw)
To: Merlin Moncure <merlin.moncure@rcsonline.com>; +Cc: Stephan Szabo <sszabo@megazone.bigpanda.com>; pgsql-performance; pgsql-hackers@postgresql.org
"Merlin Moncure" <merlin.moncure@rcsonline.com> writes:
> Here is the actual query:
> select lock_cuid(id), *
> ...
> order by wcl_vin_no, wcl_claim_no, id
> limit 1
Looks like Stephan made the right guess.
Logically the LIMIT executes after the ORDER BY, so the sorted result
has to be formed completely. The fact that we are able to optimize
this in some cases does not represent a promise that we can do it in
all cases. Ergo, it's not a bug.
regards, tom lane
^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2004-09-30 15:02 UTC | newest]
Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2004-09-30 13:45 spurious function execution in prepared statements. Merlin Moncure <merlin.moncure@rcsonline.com>
2004-09-30 14:13 ` Michael Adler <adler@pobox.com>
2004-09-30 14:45 ` Stephan Szabo <sszabo@megazone.bigpanda.com>
2004-09-30 14:54 ` Tom Lane <tgl@sss.pgh.pa.us>
2004-09-30 14:19 Re: [HACKERS] spurious function execution in prepared statements. Merlin Moncure <merlin.moncure@rcsonline.com>
2004-09-30 14:58 Re: spurious function execution in prepared statements. Merlin Moncure <merlin.moncure@rcsonline.com>
2004-09-30 15:02 ` Tom Lane <tgl@sss.pgh.pa.us>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox