Received: from maia.hub.org (maia-5.hub.org [200.46.204.29]) by mail.postgresql.org (Postfix) with ESMTP id DA615B5DDCF for ; Thu, 3 Nov 2011 15:50:06 -0300 (ADT) Received: from mail.postgresql.org ([200.46.204.86]) by maia.hub.org (mx1.hub.org [200.46.204.29]) (amavisd-maia, port 10024) with ESMTP id 35171-03-3 for ; Thu, 3 Nov 2011 18:49:59 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0-rc2 Received: from mail-vw0-f46.google.com (mail-vw0-f46.google.com [209.85.212.46]) by mail.postgresql.org (Postfix) with ESMTP id AE2FBB5DDD0 for ; Thu, 3 Nov 2011 15:49:58 -0300 (ADT) Received: by vws1 with SMTP id 1so1198206vws.19 for ; Thu, 03 Nov 2011 11:49:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type; bh=MEXC0JKdQSOSm6+kEkOBUQW0xrxrj8QGVRTpHVlUrYc=; b=sKKzsir4QXoT31OxRkpHxwxt/NLyQ1Vc1pTVGEGH/50tcpjeAcXvsfNH1G1sY5uhDu lU8LTH3ESlNxzIBBZU6CJXBE8/Jlx7EJMxLcBYz9FXYo2BsmrYMMQck1hLwFLp2ojLv3 SVT5jbeNEG4+DqbCj+FXQDhg0/JgCJaDAcnTI= Received: by 10.52.114.232 with SMTP id jj8mr10890865vdb.94.1320346198276; Thu, 03 Nov 2011 11:49:58 -0700 (PDT) Received: from jay.local (c-24-63-250-125.hsd1.ma.comcast.net. [24.63.250.125]) by mx.google.com with ESMTPS id l1sm9086780vdi.0.2011.11.03.11.49.57 (version=SSLv3 cipher=OTHER); Thu, 03 Nov 2011 11:49:57 -0700 (PDT) Message-ID: <4EB2E251.90906@gmail.com> Date: Thu, 03 Nov 2011 14:49:53 -0400 From: Jay Levitt User-Agent: Postbox 3.0.0 (Macintosh/20111031) MIME-Version: 1.0 To: Tom Lane CC: pgsql-performance@postgresql.org Subject: Re: Predicates not getting pushed into SQL function? References: <4EB2D3CB.7020708@gmail.com> <6646.1320345676@sss.pgh.pa.us> In-Reply-To: <6646.1320345676@sss.pgh.pa.us> Content-Type: multipart/related; boundary="------------010002070706050205040200" X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=-1.175 tagged_above=-10 required=5 tests=BAYES_00=-1.9, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, MIME_HTML_ONLY=0.723 X-Spam-Level: X-Archive-Number: 201111/66 X-Sequence-Number: 45577 This is a multi-part message in MIME format. --------------010002070706050205040200 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit What other info can I provide?  id is int, gender is varchar(255), and it's happening on 9.0.4...
Tom Lane
November 3, 2011 2:41 PM
Jay Levitt <jay.levitt@gmail.com> writes:
I'm confused.  I have a now-trivial SQL function that, unrestricted, would 
scan my whole users table.  When I paste the body of the function as a 
subquery and restrict it to one row, it only produces one row.  When I paste 
the body of the function into a view and restrict it to one row, it produces 
one row.  But when I put it in a SQL function... it scans the whole users 
table and then throws the other rows away.

I thought SQL functions were generally inline-able, push-down-able, etc.

inline-able, yes, but if they're not inlined you don't get any such
thing as pushdown of external conditions into the function body.
A non-inlined function is a black box.

The interesting question here is why the function doesn't get inlined
into the calling query.  You got the obvious showstoppers: it has a
SETOF result, it's not volatile, nor strict.  The only other possibility
I can see offhand is that there's some sort of result datatype mismatch,
but you've not provided enough info to be sure about that.

			regards, tom lane
Jay Levitt
November 3, 2011 1:47 PM
I'm confused.  I have a now-trivial SQL function that, unrestricted, would scan my whole users table.  When I paste the body of the function as a subquery and restrict it to one row, it only produces one row.  When I paste the body of the function into a view and restrict it to one row, it produces one row.  But when I put it in a SQL function... it scans the whole users table and then throws the other rows away.

I thought SQL functions were generally inline-able, push-down-able, etc.  As a workaround, I can put my WHERE clause inside the function and pass it parameters, but that feels ugly, and it won't help for things like resticting via JOINs.  The real function needs parameters, so I can't use it as a view.  Are there better workarounds?

I suspect the problem is (something like) the planner doesn't realize the function will produce a variable number of rows; I can specify COST or ROWS, but they're both fixed values.

Pretty-printed function and explain analyze results:

https://gist.github.com/1336963

In ASCII for web-haters and posterity:

-- THE OVERLY SIMPLIFIED FUNCTION

create or replace function matcher()
returns table(user_id int, match int) as $$

    select o.user_id, 1 as match
    from (
      select u.id as user_id, u.gender
      from users as u
    ) as o
    cross join
    (
      select u.id as user_id, u.gender
      from users as u
      where u.id = 1
    ) as my;

$$ language sql stable;

-- WHEN I CALL IT AS A FUNCTION

select * from matcher() where user_id = 2;

LOG:  duration: 1.242 ms  plan:
    Query Text:

        select o.user_id, 1 as match
        from (
          select u.id as user_id, u.gender
          from users as u
        ) as o
        cross join
        (
          select u.id as user_id, u.gender
          from users as u
          where u.id = 1
        ) as my;


    Nested Loop  (cost=0.00..118.39 rows=1656 width=4) (actual time=0.022..0.888 rows=1613 loops=1)
      Output: u.id, 1
      ->  Index Scan using users_pkey on public.users u  (cost=0.00..8.27 rows=1 width=0) (actual time=0.013..0.015 rows=1 loops=1)
            Index Cond: (u.id = 1)
      ->  Seq Scan on public.users u  (cost=0.00..93.56 rows=1656 width=4) (actual time=0.004..0.479 rows=1613 loops=1)
            Output: u.id
CONTEXT:  SQL function "matcher" statement 1
LOG:  duration: 1.951 ms  plan:
    Query Text: select * from matcher() where user_id = 2;
    Function Scan on public.matcher  (cost=0.25..12.75 rows=5 width=8) (actual time=1.687..1.940 rows=1 loops=1)
      Output: user_id, match
      Filter: (matcher.user_id = 2)

-- WHEN I CALL IT AS A SUBQUERY

select * from
(
  select o.user_id, 1 as match
  from (
    select u.id as user_id, u.gender
    from users as u
  ) as o
  cross join
  (
    select u.id as user_id, u.gender
    from users as u
    where u.id = 1
  ) as my
) as matcher
where user_id = 2;

LOG:  duration: 0.044 ms  plan:
    Query Text: select * from
    (
      select o.user_id, 1 as match
      from (
        select u.id as user_id, u.gender
        from users as u
      ) as o
      cross join
      (
        select u.id as user_id, u.gender
        from users as u
        where u.id = 1
      ) as my
    ) as matcher
    where user_id = 2;
    Nested Loop  (cost=0.00..16.55 rows=1 width=4) (actual time=0.028..0.031 rows=1 loops=1)
      Output: u.id, 1
      ->  Index Scan using users_pkey on public.users u  (cost=0.00..8.27 rows=1 width=4) (actual time=0.021..0.022 rows=1 loops=1)
            Output: u.id
            Index Cond: (u.id = 2)
      ->  Index Scan using users_pkey on public.users u  (cost=0.00..8.27 rows=1 width=0) (actual time=0.004..0.006 rows=1 loops=1)
            Index Cond: (u.id = 1)

-- WHEN I CALL IT AS A VIEW

create view matchview as
select o.user_id, 1 as match
  from (
    select u.id as user_id, u.gender
    from users as u
  ) as o
  cross join
  (
    select u.id as user_id, u.gender
    from users as u
    where u.id = 1
  ) as my;

select * from matchview where user_id = 2;


LOG:  duration: 0.044 ms  plan:
    Query Text: select * from matchview where user_id = 2;
    Nested Loop  (cost=0.00..16.55 rows=1 width=4) (actual time=0.028..0.031 rows=1 loops=1)
      Output: u.id, 1
      ->  Index Scan using users_pkey on public.users u  (cost=0.00..8.27 rows=1 width=4) (actual time=0.021..0.022 rows=1 loops=1)
            Output: u.id
            Index Cond: (u.id = 2)
      ->  Index Scan using users_pkey on public.users u  (cost=0.00..8.27 rows=1 width=0) (actual time=0.005..0.007 rows=1 loops=1)
            Index Cond: (u.id = 1)



--------------010002070706050205040200 Content-Type: image/jpeg; x-apple-mail-type=stationery; name="compose-unknown-contact.jpg" Content-Transfer-Encoding: base64 Content-ID: Content-Disposition: inline; filename="compose-unknown-contact.jpg" /9j/4AAQSkZJRgABAQEARwBHAAD/2wBDAAEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEC AQEBAQEBAgICAgICAgICAgICAgICAgICAgICAgICAgICAgL/2wBDAQEBAQEBAQICAgICAgIC AgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgL/wAAR CAAZABkDAREAAhEBAxEB/8QAGAAAAwEBAAAAAAAAAAAAAAAABgcICQr/xAA0EAABAwMCAgUK BwAAAAAAAAACAQMEBQYRABITIQcUMUF2CBUXIjI2N0JRtVRWkZOV0dL/xAAYAQEAAwEAAAAA AAAAAAAAAAADAAEEAv/EACQRAAICAAQGAwAAAAAAAAAAAAABAhEDMrHREyExM0FxgfDx/9oA DAMBAAIRAxEAPwDuEt+gW/ULet6oVC3rfqNQqFv0OfPn1GhUqfOmzZtKZlS5UqZMaNwzNwiJ VIl7eXLCaZIGwBl3TY8epPx2+jy2ZNPjvkwc9uhW8j7nCPhvOsQliYIeS7cvCpp8o50qwrC4 v3lsNSDbdmTEhvs2tahxpfV3WnmbbozJEw/gwdadbYExVRXKEKoSdvJcaOSqxE7/AAiX0gXx +a69/JSf9alIlste0VzaNpeFrcT9KKymotyiaZ0KRCnzacoE7Kjzn4gi2KqUh3jqDHDHv4mR UfruTWlMzlVUKIVNp9GguEJnAh0+IZjyAiisgyRDnu5azS8miKqjOTVkKqS/psG37fo1Fbab eg25b8eZPeFJBBJSjMG5HjMeyihnaauZwe4OGiju13GAcpOwBeN+U8/IkGbsiS8b7ryogmbz hbyc9REROfZhERO5ETShjPtvpGqTUyLErytS4siSwx5x2tRH4hPOI0DkjZtaJtFxuVEbIUUi yeNujlBUJGbJN6nM/Cyf2Hf60YgjvKA+NPSP4gT7axpcPtr51YWJnYn9dnAQWl722p4ot37y zqnlfp6FrqbwawG8/9k= --------------010002070706050205040200--