pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedGeneral advice on user functions
4+ messages / 3 participants
[nested] [flat]
* General advice on user functions
@ 2007-02-21 18:33 Dan Harris <fbsd@drivefaster.net>
0 siblings, 2 replies; 4+ messages in thread
From: Dan Harris @ 2007-02-21 18:33 UTC (permalink / raw)
To: pgsql-performance
I have a new task of automating the export of a very complex Crystal
Report. One thing I have learned in the last 36 hours is that the
export process to PDF is really, really, slooww..
Anyway, that is none of your concern. But, I am thinking that I can
somehow utilize some of PG's strengths to work around the bottleneck in
Crystal. The main problem seems to be that tens of thousands of rows of
data must be summarized in the report and calculations made. Based on
my recent experience, I'd say that this task would be better suited to
PG than relying on Crystal Reports to do the summarizing.
The difficulty I'm having is that the data needed is from about 50
different "snapshots" of counts over time. The queries are very simple,
however I believe I am going to need to combine all of these queries
into a single function that runs all 50 and then returns just the
count(*) of each as a separate "column" in a single row.
I have been Googling for hours and reading about PL/pgsql functions in
the PG docs and I have yet to find examples that returns multiple items
in a single row. I have seen cases that return "sets of", but that
appears to be returning multiple rows, not columns. Maybe this I'm
barking up the wrong tree?
Here's the gist of what I need to do:
1) query count of rows that occurred between 14 months ago and 12 months
ago for a given criteria, then count the rows that occurred between 2
months ago and current. Repeat for 50 different where clauses.
2) return each count(*) as a "column" so that in the end I can say:
select count_everything( ending_date );
and have it return to me:
count_a_lastyear count_a_last60 count_b_lastyear count_b_last60
---------------- -------------- ---------------- --------------
100 150 200 250
I'm not even sure if a function is what I'm after, maybe this can be
done in a view? I am embarrassed to ask something that seems like it
should be easy, but some key piece of knowledge is escaping me on this.
I don't expect someone to write this for me, I just need a nudge in the
right direction and maybe a URL or two to get me started.
Thank you for reading this far.
-Dan
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: General advice on user functions
@ 2007-02-21 19:05 Albert Cervera Areny <albert@sedifa.com>
parent: Dan Harris <fbsd@drivefaster.net>
1 sibling, 1 reply; 4+ messages in thread
From: Albert Cervera Areny @ 2007-02-21 19:05 UTC (permalink / raw)
To: pgsql-performance
Hi Dan,
you may take a look at the crosstab contrib module. There you can find a
function that can convert your rows into columns. However, you can also use
the manual approach, as crosstab has its limitations too.
You can create a TYPE that has all the columns you need, you create a
function that fills and returns this newly created TYPE. Of course the type
will have all those 50 fields defined, so it's boring, but should work. (Take
a look at
http://www.postgresql.org/docs/8.2/interactive/sql-createtype.html).
A Dimecres 21 Febrer 2007 19:33, Dan Harris va escriure:
> I have a new task of automating the export of a very complex Crystal
> Report. One thing I have learned in the last 36 hours is that the
> export process to PDF is really, really, slooww..
>
> Anyway, that is none of your concern. But, I am thinking that I can
> somehow utilize some of PG's strengths to work around the bottleneck in
> Crystal. The main problem seems to be that tens of thousands of rows of
> data must be summarized in the report and calculations made. Based on
> my recent experience, I'd say that this task would be better suited to
> PG than relying on Crystal Reports to do the summarizing.
>
> The difficulty I'm having is that the data needed is from about 50
> different "snapshots" of counts over time. The queries are very simple,
> however I believe I am going to need to combine all of these queries
> into a single function that runs all 50 and then returns just the
> count(*) of each as a separate "column" in a single row.
>
> I have been Googling for hours and reading about PL/pgsql functions in
> the PG docs and I have yet to find examples that returns multiple items
> in a single row. I have seen cases that return "sets of", but that
> appears to be returning multiple rows, not columns. Maybe this I'm
> barking up the wrong tree?
>
> Here's the gist of what I need to do:
>
> 1) query count of rows that occurred between 14 months ago and 12 months
> ago for a given criteria, then count the rows that occurred between 2
> months ago and current. Repeat for 50 different where clauses.
>
> 2) return each count(*) as a "column" so that in the end I can say:
>
> select count_everything( ending_date );
>
> and have it return to me:
>
> count_a_lastyear count_a_last60 count_b_lastyear count_b_last60
> ---------------- -------------- ---------------- --------------
> 100 150 200 250
>
> I'm not even sure if a function is what I'm after, maybe this can be
> done in a view? I am embarrassed to ask something that seems like it
> should be easy, but some key piece of knowledge is escaping me on this.
>
> I don't expect someone to write this for me, I just need a nudge in the
> right direction and maybe a URL or two to get me started.
>
> Thank you for reading this far.
>
> -Dan
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Have you searched our list archives?
>
> http://archives.postgresql.org
--
Albert Cervera Areny
Dept. Informàtica Sedifa, S.L.
Av. Can Bordoll, 149
08202 - Sabadell (Barcelona)
Tel. 93 715 51 11
Fax. 93 715 51 12
====================================================================
........................ AVISO LEGAL ............................
La presente comunicación y sus anexos tiene como destinatario la
persona a la que va dirigida, por lo que si usted lo recibe
por error debe notificarlo al remitente y eliminarlo de su
sistema, no pudiendo utilizarlo, total o parcialmente, para
ningún fin. Su contenido puede tener información confidencial o
protegida legalmente y únicamente expresa la opinión del
remitente. El uso del correo electrónico vía Internet no
permite asegurar ni la confidencialidad de los mensajes
ni su correcta recepción. En el caso de que el
destinatario no consintiera la utilización del correo electrónico,
deberá ponerlo en nuestro conocimiento inmediatamente.
====================================================================
........................... DISCLAIMER .............................
This message and its attachments are intended exclusively for the
named addressee. If you receive this message in error, please
immediately delete it from your system and notify the sender. You
may not use this message or any part of it for any purpose.
The message may contain information that is confidential or
protected by law, and any opinions expressed are those of the
individual sender. Internet e-mail guarantees neither the
confidentiality nor the proper receipt of the message sent.
If the addressee of this message does not consent to the use
of internet e-mail, please inform us inmmediately.
====================================================================
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: General advice on user functions
@ 2007-02-21 19:30 Merlin Moncure <mmoncure@gmail.com>
parent: Dan Harris <fbsd@drivefaster.net>
1 sibling, 0 replies; 4+ messages in thread
From: Merlin Moncure @ 2007-02-21 19:30 UTC (permalink / raw)
To: Dan Harris <fbsd@drivefaster.net>; +Cc: pgsql-performance
On 2/21/07, Dan Harris <fbsd@drivefaster.net> wrote:
> I have a new task of automating the export of a very complex Crystal
> Report. One thing I have learned in the last 36 hours is that the
> export process to PDF is really, really, slooww..
>
> Anyway, that is none of your concern. But, I am thinking that I can
> somehow utilize some of PG's strengths to work around the bottleneck in
> Crystal. The main problem seems to be that tens of thousands of rows of
> data must be summarized in the report and calculations made. Based on
> my recent experience, I'd say that this task would be better suited to
> PG than relying on Crystal Reports to do the summarizing.
>
> The difficulty I'm having is that the data needed is from about 50
> different "snapshots" of counts over time. The queries are very simple,
> however I believe I am going to need to combine all of these queries
> into a single function that runs all 50 and then returns just the
> count(*) of each as a separate "column" in a single row.
>
> I have been Googling for hours and reading about PL/pgsql functions in
> the PG docs and I have yet to find examples that returns multiple items
> in a single row. I have seen cases that return "sets of", but that
> appears to be returning multiple rows, not columns. Maybe this I'm
> barking up the wrong tree?
>
> Here's the gist of what I need to do:
>
> 1) query count of rows that occurred between 14 months ago and 12 months
> ago for a given criteria, then count the rows that occurred between 2
> months ago and current. Repeat for 50 different where clauses.
>
> 2) return each count(*) as a "column" so that in the end I can say:
>
> select count_everything( ending_date );
>
> and have it return to me:
>
> count_a_lastyear count_a_last60 count_b_lastyear count_b_last60
> ---------------- -------------- ---------------- --------------
> 100 150 200 250
>
> I'm not even sure if a function is what I'm after, maybe this can be
> done in a view? I am embarrassed to ask something that seems like it
> should be easy, but some key piece of knowledge is escaping me on this.
this could be be done in a view, a function, or a view function combo.
you can select multiple counts at once like this:
select (select count(*) from foo) as foo, (select count(*) from bar) as bar;
but this may not be appropriate in some cases where something complex
is going on. you may certainly return multiple columns from a single
call using one of two methods:
* out parameters (8.1+)
* custom type
both of which basically return a record instead of a scalar. any
function call can be wrapped in a view which can be as simple as
create view foo as select * from my_count_proc();
this is especially advised if you want to float input parameters over
a table and also filter the inputs via 'where'.
merlin
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: General advice on user functions
@ 2007-02-21 21:40 Dan Harris <fbsd@drivefaster.net>
parent: Albert Cervera Areny <albert@sedifa.com>
0 siblings, 0 replies; 4+ messages in thread
From: Dan Harris @ 2007-02-21 21:40 UTC (permalink / raw)
To: pgsql-performance
Thank you all for your ideas. I appreciate the quick response.
-Dan
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2007-02-21 21:40 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2007-02-21 18:33 General advice on user functions Dan Harris <fbsd@drivefaster.net>
2007-02-21 19:05 ` Albert Cervera Areny <albert@sedifa.com>
2007-02-21 21:40 ` Dan Harris <fbsd@drivefaster.net>
2007-02-21 19:30 ` Merlin Moncure <mmoncure@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox