public inbox for [email protected]
help / color / mirror / Atom feedRe: Using Expanded Objects other than Arrays from plpgsql
2+ messages / 2 participants
[nested] [flat]
* Re: Using Expanded Objects other than Arrays from plpgsql
@ 2024-10-20 17:13 Tom Lane <[email protected]>
0 siblings, 1 reply; 2+ messages in thread
From: Tom Lane @ 2024-10-20 17:13 UTC (permalink / raw)
To: Michel Pelletier <[email protected]>; +Cc: pgsql-general <[email protected]>
Michel Pelletier <[email protected]> writes:
> I found this thread from the original path implementation from Tom Lane in
> 2015:
> https://www.postgresql.org/message-id/E1Ysvgz-0000s0-DP%40gemulon.postgresql.org
>> In this initial implementation, a few heuristics have been hard-wired
>> into plpgsql to improve performance for arrays that are stored in
>> plpgsql variables. We would like to generalize those hacks so that
>> other datatypes can obtain similar improvements, but figuring out some
>> appropriate APIs is left as a task for future work.
Yeah, we thought that it wouldn't be appropriate to try to design
general APIs till we had more kinds of expandable objects. Maybe
now is a good time to push forward on that.
> My first thought was to add a flag to CREATE TYPE like "EXPANDED = true" or
> some other better name that indicates that the object can be safely taken
> ownership of in its expanded state and not copied.
Isn't that inherent in the notion of R/W vs R/O expanded-object
pointers?
> And then there is just removing the existing restriction on arrays only.
> Is any other expanded object out there really interested in being
> flattened/expanded over and over again?
I'm not sure. It seems certain that if the object is already expanded
(either R/W or R/O), the paths for that in plpgsql_exec_function could
be taken regardless of its specific type. The thing that is debatable
is "if the object is in a flat state, should we forcibly expand it
here?". That could be a win if the function later does object
accesses that would benefit --- but it might never do so. We chose
to always expand arrays, and we've gotten little pushback on that,
but the tradeoffs might be different for other sorts of expanded
objects.
The other problem is that plpgsql only knows how to do such expansion
for arrays, and it's not obvious how to extend that part.
But it seems like we could get an easy win by adjusting
plpgsql_exec_function along the lines of
l. 549:
- if (!var->isnull && var->datatype->typisarray)
+ if (!var->isnull)
l. 564:
- else
+ else if (var->datatype->typisarray)
How far does that improve matters for you?
The comment above line 549 cross-references exec_assign_value,
but it looks like that's already set up to be similarly type-agnostic
about values that are already expanded.
regards, tom lane
^ permalink raw reply [nested|flat] 2+ messages in thread
* Re: Using Expanded Objects other than Arrays from plpgsql
@ 2024-10-20 18:25 Michel Pelletier <[email protected]>
parent: Tom Lane <[email protected]>
0 siblings, 0 replies; 2+ messages in thread
From: Michel Pelletier @ 2024-10-20 18:25 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: pgsql-general <[email protected]>
On Sun, Oct 20, 2024 at 10:13 AM Tom Lane <[email protected]> wrote:
> Michel Pelletier <[email protected]> writes:
> > I found this thread from the original path implementation from Tom Lane
> in
>
> >> appropriate APIs is left as a task for future work.
>
> Yeah, we thought that it wouldn't be appropriate to try to design
> general APIs till we had more kinds of expandable objects. Maybe
> now is a good time to push forward on that.
>
Great, I'm happy to be involved!
> My first thought was to add a flag to CREATE TYPE like "EXPANDED = true"
> or
> > some other better name that indicates that the object can be safely taken
> > ownership of in its expanded state and not copied.
>
> Isn't that inherent in the notion of R/W vs R/O expanded-object
> pointers?
>
I'm not sure I'm qualified enough to agree with you but I see your point.
> > And then there is just removing the existing restriction on arrays only.
> > Is any other expanded object out there really interested in being
> > flattened/expanded over and over again?
>
> I'm not sure. It seems certain that if the object is already expanded
> (either R/W or R/O), the paths for that in plpgsql_exec_function could
> be taken regardless of its specific type.
Agreed.
> The thing that is debatable
> is "if the object is in a flat state, should we forcibly expand it
> here?". That could be a win if the function later does object
> accesses that would benefit --- but it might never do so. We chose
> to always expand arrays, and we've gotten little pushback on that,
> but the tradeoffs might be different for other sorts of expanded
> objects.
>
The OneSparse objects will always expand themselves on first use via a
DatumGetExpandedArray like macro wrapper, there is no run-time benefit to
their flat representation, so I'm fine with not forcing their expansion
ahead of time, but once I expand it I'd like it to stay expanded until the
function returns (as much as possible) the serialization costs for very
large sparse matrices is heavy.
The other problem is that plpgsql only knows how to do such expansion
> for arrays, and it's not obvious how to extend that part.
>
Perhaps a third member function for ExpandedObjectMethods that formalizes
the expansion interface like found in DatumGetExpandedArray? I closely
follow that same pattern in my code.
>
> But it seems like we could get an easy win by adjusting
> plpgsql_exec_function along the lines of
> ...
>
> How far does that improve matters for you?
>
I'll give it a try in my local benchmarking code and get back to you, thank
you for the fast reply and the insight!
-Michel
^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2024-10-20 18:25 UTC | newest]
Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2024-10-20 17:13 Re: Using Expanded Objects other than Arrays from plpgsql Tom Lane <[email protected]>
2024-10-20 18:25 ` Michel Pelletier <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox