pg.ddx.io pgsql-hackers@postgresql.org mailing list archive
help / color / mirror / Atom feedECPG FETCH readahead, was: Re: ECPG fixes
8+ messages / 4 participants
[nested] [flat]
* ECPG FETCH readahead, was: Re: ECPG fixes
@ 2013-11-20 14:25 Boszormenyi Zoltan <zb@cybertec.at>
0 siblings, 1 reply; 8+ messages in thread
From: Boszormenyi Zoltan @ 2013-11-20 14:25 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@2ndquadrant.com>; Michael Meskes <meskes@postgresql.org>; +Cc: Noah Misch <noah@leadboat.com>; Peter Eisentraut <peter_e@gmx.net>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers; Heikki Linnakangas <hlinnakangas@vmware.com>; Bruce Momjian <bruce@momjian.us>; Hans-Jürgen Schönig <hs@cybertec.at>
2013-11-20 14:41 keltezéssel, Boszormenyi Zoltan írta:
> 2013-11-12 07:15 keltezéssel, Boszormenyi Zoltan írta:
>> 2013-11-12 07:01 keltezéssel, Noah Misch írta:
>>> On Mon, Nov 11, 2013 at 10:17:54AM +0100, Boszormenyi Zoltan wrote:
>>>> The old contents of my GIT repository was removed so you need to
>>>> clone it fresh. https://github.com/zboszor/ecpg-readahead.git
>>>> I won't post the humongous patch again, since sending a 90KB
>>>> compressed file to everyone on the list is rude.
>>> Patches of that weight show up on a regular basis. I don't think it's rude.
>>
>> OK, here it is.
>
> ...
> Subsequent patches will come as reply to this email.
$SUBJECT, the real thing. Needs all previous patches.
Compared the the previous incarnation, a lot of debugging
ecpg_log() lines are removed. The newly introduced regression
tests have much smaller .stderr outputs because of this,
the patch is about 800K smaller in total.
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Attachments:
[application/x-tar] 26.patch.gz (45.8K, ../../528CC655.2070309@cybertec.at/2-26.patch.gz)
download
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: ECPG FETCH readahead, was: Re: ECPG fixes
@ 2013-11-28 14:23 Boszormenyi Zoltan <zb@cybertec.at>
parent: Boszormenyi Zoltan <zb@cybertec.at>
0 siblings, 1 reply; 8+ messages in thread
From: Boszormenyi Zoltan @ 2013-11-28 14:23 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@2ndquadrant.com>; Michael Meskes <meskes@postgresql.org>; +Cc: Noah Misch <noah@leadboat.com>; Peter Eisentraut <peter_e@gmx.net>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers; Heikki Linnakangas <hlinnakangas@vmware.com>; Bruce Momjian <bruce@momjian.us>; Hans-Jürgen Schönig <hs@cybertec.at>
2013-11-20 15:25 keltezéssel, Boszormenyi Zoltan írta:
> 2013-11-20 14:41 keltezéssel, Boszormenyi Zoltan írta:
>> 2013-11-12 07:15 keltezéssel, Boszormenyi Zoltan írta:
>>> 2013-11-12 07:01 keltezéssel, Noah Misch írta:
>>>> On Mon, Nov 11, 2013 at 10:17:54AM +0100, Boszormenyi Zoltan wrote:
>>>>> The old contents of my GIT repository was removed so you need to
>>>>> clone it fresh. https://github.com/zboszor/ecpg-readahead.git
>>>>> I won't post the humongous patch again, since sending a 90KB
>>>>> compressed file to everyone on the list is rude.
>>>> Patches of that weight show up on a regular basis. I don't think it's rude.
>>>
>>> OK, here it is.
>>
>> ...
>> Subsequent patches will come as reply to this email.
>
> $SUBJECT, the real thing. Needs all previous patches.
> Compared the the previous incarnation, a lot of debugging
> ecpg_log() lines are removed. The newly introduced regression
> tests have much smaller .stderr outputs because of this,
> the patch is about 800K smaller in total.
Rebased after killing the patch that changed the DECLARE CURSOR command tag.
The ECPG readahead patch and all the small following patches are attached.
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Attachments:
[application/x-tar] 26.patch.gz (41.4K, ../../529751FB.40107@cybertec.at/2-26.patch.gz)
download
[application/x-tar] 27.patch.gz (5.6K, ../../529751FB.40107@cybertec.at/3-27.patch.gz)
download
[application/x-tar] 28.patch.gz (6.3K, ../../529751FB.40107@cybertec.at/4-28.patch.gz)
download
[application/x-tar] 29.patch.gz (3.4K, ../../529751FB.40107@cybertec.at/5-29.patch.gz)
download
[application/x-tar] 30.patch.gz (888B, ../../529751FB.40107@cybertec.at/6-30.patch.gz)
download
[application/x-tar] 31.patch.gz (6.3K, ../../529751FB.40107@cybertec.at/7-31.patch.gz)
download
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: ECPG FETCH readahead, was: Re: ECPG fixes
@ 2013-12-04 13:52 Boszormenyi Zoltan <zb@cybertec.at>
parent: Boszormenyi Zoltan <zb@cybertec.at>
0 siblings, 1 reply; 8+ messages in thread
From: Boszormenyi Zoltan @ 2013-12-04 13:52 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@2ndquadrant.com>; Michael Meskes <meskes@postgresql.org>; +Cc: Noah Misch <noah@leadboat.com>; Peter Eisentraut <peter_e@gmx.net>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers; Heikki Linnakangas <hlinnakangas@vmware.com>; Bruce Momjian <bruce@momjian.us>; Hans-Jürgen Schönig <hs@cybertec.at>
2013-11-28 15:23 keltezéssel, Boszormenyi Zoltan írta:
> Rebased after killing the patch that changed the DECLARE CURSOR command tag.
> The ECPG readahead patch and all the small following patches are attached.
Fixed the extra spaces that "git apply" complains about.
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Attachments:
[application/x-tar] 26.patch.gz (41.3K, ../../529F33B9.6010208@cybertec.at/2-26.patch.gz)
download
[application/x-tar] 27.patch.gz (5.6K, ../../529F33B9.6010208@cybertec.at/3-27.patch.gz)
download
[application/x-tar] 28.patch.gz (6.3K, ../../529F33B9.6010208@cybertec.at/4-28.patch.gz)
download
[application/x-tar] 29.patch.gz (3.4K, ../../529F33B9.6010208@cybertec.at/5-29.patch.gz)
download
[text/x-patch] 30.patch (1.6K, ../../529F33B9.6010208@cybertec.at/6-30.patch)
download | inline diff:
commit 29425979f229ae99fe32b8540d65919e517b1c33
Author: Böszörményi Zoltán <zb@cybertec.at>
Date: Wed Dec 4 14:27:29 2013 +0100
ECPG: During FETCH ALL, temporarily use FETCHALL_MULTIPLIER times
the current readahead window size. FETCHALL_MULTIPLIER == 1000.
No change in regression tests.
diff --git a/src/interfaces/ecpg/ecpglib/cursor.c b/src/interfaces/ecpg/ecpglib/cursor.c
index 1c24133..2d024b8 100644
--- a/src/interfaces/ecpg/ecpglib/cursor.c
+++ b/src/interfaces/ecpg/ecpglib/cursor.c
@@ -1518,6 +1518,7 @@ ecpg_cursor_fetch(struct statement *stmt, struct cursor_descriptor *cur,
long ntuples;
int step;
int64 prev_pos, next_pos, start_idx, var_index;
+ int64 old_readahead;
switch (direction)
{
@@ -1689,6 +1690,10 @@ abs_rel:
goto abs_rel;
}
+ old_readahead = cur->readahead;
+ if (fetchall)
+ cur->readahead *= FETCHALL_MULTIPLIER;
+
/*
* The direction is backward if FETCH BACKWARD ALL
* or the amount to fetch is negative.
@@ -1772,6 +1777,8 @@ abs_rel:
sqlca->sqlerrd[2] = (var_index <= LONG_MAX ? var_index : 0);
+ cur->readahead = old_readahead;
+
if (var_index == 0)
{
ecpg_raise(stmt->lineno, ECPG_NOT_FOUND, ECPG_SQLSTATE_NO_DATA, NULL);
diff --git a/src/interfaces/ecpg/ecpglib/extern.h b/src/interfaces/ecpg/ecpglib/extern.h
index 612412f..de0f3b8 100644
--- a/src/interfaces/ecpg/ecpglib/extern.h
+++ b/src/interfaces/ecpg/ecpglib/extern.h
@@ -32,6 +32,7 @@ enum ARRAY_TYPE
#define LOOP_FORWARD (1)
#define LOOP_BACKWARD (-1)
#define MAX_CACHE_MISS (3)
+#define FETCHALL_MULTIPLIER (1000)
/* A generic varchar type. */
struct ECPGgeneric_varchar
[application/x-tar] 31.patch.gz (6.3K, ../../529F33B9.6010208@cybertec.at/7-31.patch.gz)
download
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: ECPG FETCH readahead, was: Re: ECPG fixes
@ 2013-12-06 13:05 Boszormenyi Zoltan <zb@cybertec.at>
parent: Boszormenyi Zoltan <zb@cybertec.at>
0 siblings, 2 replies; 8+ messages in thread
From: Boszormenyi Zoltan @ 2013-12-06 13:05 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@2ndquadrant.com>; Michael Meskes <meskes@postgresql.org>; +Cc: Noah Misch <noah@leadboat.com>; Peter Eisentraut <peter_e@gmx.net>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers; Heikki Linnakangas <hlinnakangas@vmware.com>; Bruce Momjian <bruce@momjian.us>; Hans-Jürgen Schönig <hs@cybertec.at>; Antonin Houska <antonin.houska@gmail.com>
Rebased patches after the regression test and other details were fixed
in the infrastructure part.
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Attachments:
[application/x-tar] 26.patch.gz (41.3K, ../../52A1CB7F.70805@cybertec.at/2-26.patch.gz)
download
[application/x-tar] 27.patch.gz (5.5K, ../../52A1CB7F.70805@cybertec.at/3-27.patch.gz)
download
[application/x-tar] 28.patch.gz (6.2K, ../../52A1CB7F.70805@cybertec.at/4-28.patch.gz)
download
[application/x-tar] 29.patch.gz (3.4K, ../../52A1CB7F.70805@cybertec.at/5-29.patch.gz)
download
[text/x-patch] 30.patch (1.6K, ../../52A1CB7F.70805@cybertec.at/6-30.patch)
download | inline diff:
commit 0640bcefe4452e04417c61af3cbb70ccadf12554
Author: Böszörményi Zoltán <zb@cybertec.at>
Date: Fri Dec 6 13:49:32 2013 +0100
ECPG: During FETCH ALL, temporarily use FETCHALL_MULTIPLIER times
the current readahead window size. FETCHALL_MULTIPLIER == 1000.
No change in regression tests.
diff --git a/src/interfaces/ecpg/ecpglib/cursor.c b/src/interfaces/ecpg/ecpglib/cursor.c
index da83c92..7e231d7 100644
--- a/src/interfaces/ecpg/ecpglib/cursor.c
+++ b/src/interfaces/ecpg/ecpglib/cursor.c
@@ -1517,6 +1517,7 @@ ecpg_cursor_fetch(struct statement *stmt, struct cursor_descriptor *cur,
long ntuples;
int step;
int64 prev_pos, next_pos, start_idx, var_index;
+ int64 old_readahead;
switch (direction)
{
@@ -1688,6 +1689,10 @@ abs_rel:
goto abs_rel;
}
+ old_readahead = cur->readahead;
+ if (fetchall)
+ cur->readahead *= FETCHALL_MULTIPLIER;
+
/*
* The direction is backward if FETCH BACKWARD ALL
* or the amount to fetch is negative.
@@ -1771,6 +1776,8 @@ abs_rel:
sqlca->sqlerrd[2] = (var_index <= LONG_MAX ? var_index : 0);
+ cur->readahead = old_readahead;
+
if (var_index == 0)
{
ecpg_raise(stmt->lineno, ECPG_NOT_FOUND, ECPG_SQLSTATE_NO_DATA, NULL);
diff --git a/src/interfaces/ecpg/ecpglib/extern.h b/src/interfaces/ecpg/ecpglib/extern.h
index 612412f..de0f3b8 100644
--- a/src/interfaces/ecpg/ecpglib/extern.h
+++ b/src/interfaces/ecpg/ecpglib/extern.h
@@ -32,6 +32,7 @@ enum ARRAY_TYPE
#define LOOP_FORWARD (1)
#define LOOP_BACKWARD (-1)
#define MAX_CACHE_MISS (3)
+#define FETCHALL_MULTIPLIER (1000)
/* A generic varchar type. */
struct ECPGgeneric_varchar
[application/x-tar] 31.patch.gz (6.4K, ../../52A1CB7F.70805@cybertec.at/7-31.patch.gz)
download
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: ECPG FETCH readahead, was: Re: ECPG fixes
@ 2013-12-21 13:56 Peter Eisentraut <peter_e@gmx.net>
parent: Boszormenyi Zoltan <zb@cybertec.at>
1 sibling, 1 reply; 8+ messages in thread
From: Peter Eisentraut @ 2013-12-21 13:56 UTC (permalink / raw)
To: Boszormenyi Zoltan <zb@cybertec.at>; +Cc: Alvaro Herrera <alvherre@2ndquadrant.com>; Michael Meskes <meskes@postgresql.org>; Noah Misch <noah@leadboat.com>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers; Heikki Linnakangas <hlinnakangas@vmware.com>; Bruce Momjian <bruce@momjian.us>; Hans-Jürgen Schönig <hs@cybertec.at>; Antonin Houska <antonin.houska@gmail.com>
This patch didn't make it out of the 2013-11 commit fest. You should
move it to the next commit fest (probably with an updated patch)
before January 15th.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: ECPG FETCH readahead, was: Re: ECPG fixes
@ 2013-12-23 09:39 Boszormenyi Zoltan <zb@cybertec.at>
parent: Peter Eisentraut <peter_e@gmx.net>
0 siblings, 0 replies; 8+ messages in thread
From: Boszormenyi Zoltan @ 2013-12-23 09:39 UTC (permalink / raw)
To: Peter Eisentraut <peter_e@gmx.net>; +Cc: Alvaro Herrera <alvherre@2ndquadrant.com>; Michael Meskes <meskes@postgresql.org>; Noah Misch <noah@leadboat.com>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers; Heikki Linnakangas <hlinnakangas@vmware.com>; Bruce Momjian <bruce@momjian.us>; Hans-Jürgen Schönig <hs@cybertec.at>; Antonin Houska <antonin.houska@gmail.com>
2013-12-21 14:56 keltezéssel, Peter Eisentraut írta:
> This patch didn't make it out of the 2013-11 commit fest. You should
> move it to the next commit fest (probably with an updated patch)
> before January 15th.
Done.
Best regards,
Zoltán Böszörményi
--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: ECPG FETCH readahead, was: Re: ECPG fixes
@ 2014-01-16 22:42 Alvaro Herrera <alvherre@2ndquadrant.com>
parent: Boszormenyi Zoltan <zb@cybertec.at>
1 sibling, 1 reply; 8+ messages in thread
From: Alvaro Herrera @ 2014-01-16 22:42 UTC (permalink / raw)
To: Boszormenyi Zoltan <zb@cybertec.at>; +Cc: Michael Meskes <meskes@postgresql.org>; Noah Misch <noah@leadboat.com>; Peter Eisentraut <peter_e@gmx.net>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers; Heikki Linnakangas <hlinnakangas@vmware.com>; Bruce Momjian <bruce@momjian.us>; Hans-Jürgen Schönig <hs@cybertec.at>; Antonin Houska <antonin.houska@gmail.com>
Boszormenyi Zoltan escribió:
> Rebased patches after the regression test and other details were fixed
> in the infrastructure part.
This thread started in 2010, and various pieces have been applied
already and some others have changed in nature. Would you please post a
new patchset, containing rebased patches that still need application, in
a new email thread to be linked in the commitfest entry?
--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: ECPG FETCH readahead, was: Re: ECPG fixes
@ 2014-01-17 08:00 Boszormenyi Zoltan <zboszor@pr.hu>
parent: Alvaro Herrera <alvherre@2ndquadrant.com>
0 siblings, 0 replies; 8+ messages in thread
From: Boszormenyi Zoltan @ 2014-01-17 08:00 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@2ndquadrant.com>; +Cc: Michael Meskes <meskes@postgresql.org>; Noah Misch <noah@leadboat.com>; Peter Eisentraut <peter_e@gmx.net>; Robert Haas <robertmhaas@gmail.com>; pgsql-hackers; Heikki Linnakangas <hlinnakangas@vmware.com>; Bruce Momjian <bruce@momjian.us>; Hans-Jürgen Schönig <hs@cybertec.at>; Antonin Houska <antonin.houska@gmail.com>
2014-01-16 23:42 keltezéssel, Alvaro Herrera írta:
> Boszormenyi Zoltan escribió:
>> Rebased patches after the regression test and other details were fixed
>> in the infrastructure part.
> This thread started in 2010, and various pieces have been applied
> already and some others have changed in nature. Would you please post a
> new patchset, containing rebased patches that still need application, in
> a new email thread to be linked in the commitfest entry?
I will do that.
Best regards,
Zoltán Böszörményi
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
^ permalink raw reply [nested|flat] 8+ messages in thread
end of thread, other threads:[~2014-01-17 08:00 UTC | newest]
Thread overview: 8+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2013-11-20 14:25 ECPG FETCH readahead, was: Re: ECPG fixes Boszormenyi Zoltan <zb@cybertec.at>
2013-11-28 14:23 ` Boszormenyi Zoltan <zb@cybertec.at>
2013-12-04 13:52 ` Boszormenyi Zoltan <zb@cybertec.at>
2013-12-06 13:05 ` Boszormenyi Zoltan <zb@cybertec.at>
2013-12-21 13:56 ` Peter Eisentraut <peter_e@gmx.net>
2013-12-23 09:39 ` Boszormenyi Zoltan <zb@cybertec.at>
2014-01-16 22:42 ` Alvaro Herrera <alvherre@2ndquadrant.com>
2014-01-17 08:00 ` Boszormenyi Zoltan <zboszor@pr.hu>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox